~/.aws/config
file with different profiles:
[default]
region = us-east-1role_arn = arn:aws:iam::XXXXX:role/engineer
source_profile = default[profile john]
role_arn = arn:aws:iam::YYYYY:role/marketingsource_profile = default
[profile james]
role_arn = arn:aws:iam::ZZZZ:role/marketingsource_profile = default
This will allow you to pass --profile john
to the CLI.
However, AWS node SDK does not support the profile option. You can always verify the "current user" by:
izyaws.sh userId sts get-caller-identity
Or, from the scripting environment:
const sts = new AWS.STS();
sts.getCallerIdentity((err, data) => {
console.log(err, data);});
To work around this problem, you can use assume role:
izyaws.sh
// full access to AWS services and resources.
{"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow","Action": "*",
"Resource": "*"}
]}
You can then set
export AWSACCESSKEY_ID=
export AWSSECRETACCESS_KEY=export AWSSESSIONTOKEN=
After you are done, be sure to unset the variables by
unset AWSACCESSKEYID AWSSECRETACCESSKEY AWSSESSIONTOKEN
If you need to generate a One Time Password from the cli use
npm run otp.generate queryObject.secretStrBase32 your_secret
npm run otp.generate queryObject.secretPath ~/.aws/otpsecretnpm run otp.generate queryObject.secretPath ~/googleAccount1/otpsecret
To create an aws session automatically using the otp feature do
izy.aws "sts?newSession"
You can use the following snippet inside aws-sdk/lib/http/node.js
.
console.log('HTTP_REQUEST', JSON.stringify(httpRequest, null, 2));
var stream = http.request(options, function (httpResp) {console.log('OK-------------------------');
var str = '';
var response = httpResp;response.on('data', function (chunk) {
str += chunk;});
response.on('end', function () {console.log({
success: true,responseText: str,
status: response.statusCode,headers: response.headers
});});
return ;You can access the ssh scripts by the ssh prefix:
izy.devops "ssh?shell" MACHINE_ID
izy.devops "ssh?runx.sh" . "xeyes" izy.devops "rsync?download" . "~/Downloads/" ~/izyware/izy-idman-tools/id/x/Desktop/Downloads/izy.devops "rsync?upload" . "~/Downloads/" ~/izyware/izy-idman-tools/id/x/Desktop/Downloads/
izy.devops "rsync?nodesync" ~/plat/p/servers/machine "~/codepath" appname@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: UNPROTECTED PRIVATE KEY FILE! @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for 'private.pem' are too open.To fix this chmod to
chmod 400 private.pem
izy.aws "codebuild?check" queryObject.izyUser 86 queryObject.showLogs true queryObject.projectName myProject
Exploring codebuild setup and viewing a build projects details
npm run codebuild.projectDetails queryObject.izyUser 86 queryObject.projectName myProject
To debug your code build scripts (buildspec.*) use the provided docker Image:
docker build -f codebuild/Dockerfile -t izy-aws-codebuild:7.0 .;
You can then shell into the environment and run your commands:
docker run -e BUILDDIR=pwd
--env-file ~/dockerenv-file.list -v /var/run/docker.sock:/var/run/docker.sock -v pwd
:/sourcedirectory -it --entrypoint sh izy-aws-codebuild:7.0 -c bash;
dockershell>killanddeletedata.sh imagekeyword (i.e. sql)
Notice that if you dont have a env file or AWS credentials are not available as environment variables you share your .aws folder:
-v echo ~/.aws
:/root/.aws
If your application generates or uses docker containers as part of its CICD, you will need to use privilaged mode inside codebuild. Notice that the filesystem/volume mapping and network interfaces addresses can get confusing in this case. The most common issue reported on MacOS platforms vs Linux (Codebuild) is that the docker-compose volume mapping (using the volumes block) on Mac will only work if full path to host operating system is provided. Therefore, it is recommended to use the BUILD_DIR
variable.
You may use the dockertools/serviceprobe docker image to troubleshoot these issues. Refer to the Dockerfile for more information and examples about how to use the tool. Notice that if you are using dockercompose you can specifiy the following in the environment section:
ENV CONTAINER_CONTEXT: Service Probe
ENV SLEEP_SECONDS: 5ENV CMDTORUN: /usr/bin/nc -vz localhost 3306
Deep clone the app directory:
rsync -rv apps/static-website/ myfolder
cd myfolder; Update the variables (domain, aws credentials, etc.) inside thevi terraform_config.sh
Make sure that you dont use the naked (apex) domain name and use a prefix (www, etc.). DNS protocol does not support CNAME records or 301 redirects for the apex record. See the section regarding AWS Quirks below to learn how to redirect the apex to your subdomain (domain -> www.domain).
Shell into the image:
docker run -v pwd
:/izyhostdir -it --entrypoint sh hashicorp/terraform:latest
Once inside the image, do:
clear; cd /izyhostdir; source ./terraform_config.sh; terraform apply -auto-approve;
The output should print the list of environment variables that will be used below:
export AWSCLOUDFRONTDISTRIBUTIONDOMAINNAME=GRABFROMTERRAFORM_OUTPUT;
export AWSCLOUDFRONTDISTRIBUTIONID=GRABFROMTERRAFORMOUTPUT;export AWSS3BUCKETID=GRABFROMTERRAFORMOUTPUT;
You can push your files to the s3 bucket:
izyaws.sh
Make sure to invalidate the cloudfront distribution after the push
izyaws.sh
* when this is being created for the firsttime, you might get a "Missing required argument" error, followed by The argument "origin.0.domain_name" is required, but no definition was found.
. rerunning the plan will address this.
deleteall.sh
If you have a stateful container (i.e. mysql), you should stop and delete its state by
killanddeletedata.sh grepStr (mysql)
To build runtimes
docker build -f dockertools/nodejs/Dockerfile -t izy-nodejs:16.0 .;
docker run -it --entrypoint sh izy-nodejs:16.0 -c bash;* May not work on some device since Permissions-Policy: screen-wake-lock=
may be required.
* useful for exploring codebuild setup and viewing a build projects details in AWS CodeBuild
* utilize callpretty for better output* add verbose logging
github: https://github.com/izyware/devops