Is it possible/recommended to use `sam build` in AWS CodeBuild?
This question spun out of this one. Now that I have a better understanding of what was going wrong there, and a workable, if imperfect, solution, I'm submitting a more focused follow-up (I'm still something of a novice at StackOverflow - please let me know if this contravenes etiquette, and I should follow-up on the original).
This page suggests that "You use AWS CodeBuild to build, locally test, and package your serverless application". However, when I include a sam build command in my buildspec.yml, I get the following log output, suggesting that sam is not installed on CodeBuild images:
[Container] 2018/12/31 11:41:49 Running command sam build --use-container
sh: 1: sam: not found
[Container] 2018/12/31 11:41:49 Command did not exit successfully sam build --use-container exit status 127
[Container] 2018/12/31 11:41:49 Phase complete: BUILD Success: false
[Container] 2018/12/31 11:41:49 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: sam build --use-container. Reason: exit status 127
Furthermore, if I install SAM with pip install aws-sam-cli, running sam build --use-container in CodeBuild gives an error. sam build itself succeeds, but since it doesn't install test dependencies, I'd still need to use pip install -r tests/requirements-test.txt -t . to be able to run tests in CodeBuild. Moreover, this suggests that --use-container is required for "packages that have natively compiled programs").
This makes me wonder whether I'm trying to do something wrong. What's the recommended way of building SAM services in a CI/CD workflow on AWS?
amazon-web-services aws-codebuild aws-serverless
add a comment |
This question spun out of this one. Now that I have a better understanding of what was going wrong there, and a workable, if imperfect, solution, I'm submitting a more focused follow-up (I'm still something of a novice at StackOverflow - please let me know if this contravenes etiquette, and I should follow-up on the original).
This page suggests that "You use AWS CodeBuild to build, locally test, and package your serverless application". However, when I include a sam build command in my buildspec.yml, I get the following log output, suggesting that sam is not installed on CodeBuild images:
[Container] 2018/12/31 11:41:49 Running command sam build --use-container
sh: 1: sam: not found
[Container] 2018/12/31 11:41:49 Command did not exit successfully sam build --use-container exit status 127
[Container] 2018/12/31 11:41:49 Phase complete: BUILD Success: false
[Container] 2018/12/31 11:41:49 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: sam build --use-container. Reason: exit status 127
Furthermore, if I install SAM with pip install aws-sam-cli, running sam build --use-container in CodeBuild gives an error. sam build itself succeeds, but since it doesn't install test dependencies, I'd still need to use pip install -r tests/requirements-test.txt -t . to be able to run tests in CodeBuild. Moreover, this suggests that --use-container is required for "packages that have natively compiled programs").
This makes me wonder whether I'm trying to do something wrong. What's the recommended way of building SAM services in a CI/CD workflow on AWS?
amazon-web-services aws-codebuild aws-serverless
add a comment |
This question spun out of this one. Now that I have a better understanding of what was going wrong there, and a workable, if imperfect, solution, I'm submitting a more focused follow-up (I'm still something of a novice at StackOverflow - please let me know if this contravenes etiquette, and I should follow-up on the original).
This page suggests that "You use AWS CodeBuild to build, locally test, and package your serverless application". However, when I include a sam build command in my buildspec.yml, I get the following log output, suggesting that sam is not installed on CodeBuild images:
[Container] 2018/12/31 11:41:49 Running command sam build --use-container
sh: 1: sam: not found
[Container] 2018/12/31 11:41:49 Command did not exit successfully sam build --use-container exit status 127
[Container] 2018/12/31 11:41:49 Phase complete: BUILD Success: false
[Container] 2018/12/31 11:41:49 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: sam build --use-container. Reason: exit status 127
Furthermore, if I install SAM with pip install aws-sam-cli, running sam build --use-container in CodeBuild gives an error. sam build itself succeeds, but since it doesn't install test dependencies, I'd still need to use pip install -r tests/requirements-test.txt -t . to be able to run tests in CodeBuild. Moreover, this suggests that --use-container is required for "packages that have natively compiled programs").
This makes me wonder whether I'm trying to do something wrong. What's the recommended way of building SAM services in a CI/CD workflow on AWS?
amazon-web-services aws-codebuild aws-serverless
This question spun out of this one. Now that I have a better understanding of what was going wrong there, and a workable, if imperfect, solution, I'm submitting a more focused follow-up (I'm still something of a novice at StackOverflow - please let me know if this contravenes etiquette, and I should follow-up on the original).
This page suggests that "You use AWS CodeBuild to build, locally test, and package your serverless application". However, when I include a sam build command in my buildspec.yml, I get the following log output, suggesting that sam is not installed on CodeBuild images:
[Container] 2018/12/31 11:41:49 Running command sam build --use-container
sh: 1: sam: not found
[Container] 2018/12/31 11:41:49 Command did not exit successfully sam build --use-container exit status 127
[Container] 2018/12/31 11:41:49 Phase complete: BUILD Success: false
[Container] 2018/12/31 11:41:49 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: sam build --use-container. Reason: exit status 127
Furthermore, if I install SAM with pip install aws-sam-cli, running sam build --use-container in CodeBuild gives an error. sam build itself succeeds, but since it doesn't install test dependencies, I'd still need to use pip install -r tests/requirements-test.txt -t . to be able to run tests in CodeBuild. Moreover, this suggests that --use-container is required for "packages that have natively compiled programs").
This makes me wonder whether I'm trying to do something wrong. What's the recommended way of building SAM services in a CI/CD workflow on AWS?
amazon-web-services aws-codebuild aws-serverless
amazon-web-services aws-codebuild aws-serverless
edited Dec 31 '18 at 12:12
scubbo
asked Dec 31 '18 at 11:55
scubboscubbo
1,18642139
1,18642139
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
Please see below for buildspec.yaml that works for me when using AWS SAM with AWS CodeBuild, with cloudformation.yml
phases:
build:
commands:
- pip install --user aws-sam-cli
- USER_BASE_PATH=$(python -m site --user-base)
- export PATH=$PATH:$USER_BASE_PATH/bin
- sam build -t cloudformation.yml
- aws cloudformation package --template-file .aws-sam/build/template.yaml --s3-bucket <TARGET_S3_BUCKET> --output-template-file cloudformation-packaged.yaml
- aws s3 cp ./cloudformation-packaged.yaml <TARGET_S3_BUCKET>/cloudformation-packaged.yaml
In the result I get a deployment package and packaged cloudformation template in the TARGET_S3_BUCKET.
For each function in the ./src folder, I have a requirements.txt file that includes all the dependencies, but I dont run pip install -r requirements.txt manually.
Hey Lech! Thanks for the example. Given that this presumably only installs your source requirements (not Test Requirements - likerequests-mock,pytest, etc.), how do you run your tests during build? Does this setup allow you to use "packages that have natively compiled programs" (which usually requiredsam build --use-container)? Is there a reason why you're exporting thecloudformation-packaged.yamlfile to S3, rather than just referencing it directly in a "Deploy" phase of your CodePipeline?
– scubbo
Dec 31 '18 at 13:26
Hi, all good questions! :) 1. I dont do tests here, I only use CodeBuild for building. If you'd like to do tests, yes, I'd say you should run pip. 2. As for moving packaged file, potentially I guess you could do this with CodeDeploy, but since I have very specific and unusual structure of folders, I ended up doing it manually with AWS CLI :)
– Lech Migdal
Dec 31 '18 at 13:34
Got it, thanks! I'll wait to accept a more complete and/or generally applicable answer, then - but it was helpful to see someone else's setup, thank you!
– scubbo
Dec 31 '18 at 13:53
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53987204%2fis-it-possible-recommended-to-use-sam-build-in-aws-codebuild%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Please see below for buildspec.yaml that works for me when using AWS SAM with AWS CodeBuild, with cloudformation.yml
phases:
build:
commands:
- pip install --user aws-sam-cli
- USER_BASE_PATH=$(python -m site --user-base)
- export PATH=$PATH:$USER_BASE_PATH/bin
- sam build -t cloudformation.yml
- aws cloudformation package --template-file .aws-sam/build/template.yaml --s3-bucket <TARGET_S3_BUCKET> --output-template-file cloudformation-packaged.yaml
- aws s3 cp ./cloudformation-packaged.yaml <TARGET_S3_BUCKET>/cloudformation-packaged.yaml
In the result I get a deployment package and packaged cloudformation template in the TARGET_S3_BUCKET.
For each function in the ./src folder, I have a requirements.txt file that includes all the dependencies, but I dont run pip install -r requirements.txt manually.
Hey Lech! Thanks for the example. Given that this presumably only installs your source requirements (not Test Requirements - likerequests-mock,pytest, etc.), how do you run your tests during build? Does this setup allow you to use "packages that have natively compiled programs" (which usually requiredsam build --use-container)? Is there a reason why you're exporting thecloudformation-packaged.yamlfile to S3, rather than just referencing it directly in a "Deploy" phase of your CodePipeline?
– scubbo
Dec 31 '18 at 13:26
Hi, all good questions! :) 1. I dont do tests here, I only use CodeBuild for building. If you'd like to do tests, yes, I'd say you should run pip. 2. As for moving packaged file, potentially I guess you could do this with CodeDeploy, but since I have very specific and unusual structure of folders, I ended up doing it manually with AWS CLI :)
– Lech Migdal
Dec 31 '18 at 13:34
Got it, thanks! I'll wait to accept a more complete and/or generally applicable answer, then - but it was helpful to see someone else's setup, thank you!
– scubbo
Dec 31 '18 at 13:53
add a comment |
Please see below for buildspec.yaml that works for me when using AWS SAM with AWS CodeBuild, with cloudformation.yml
phases:
build:
commands:
- pip install --user aws-sam-cli
- USER_BASE_PATH=$(python -m site --user-base)
- export PATH=$PATH:$USER_BASE_PATH/bin
- sam build -t cloudformation.yml
- aws cloudformation package --template-file .aws-sam/build/template.yaml --s3-bucket <TARGET_S3_BUCKET> --output-template-file cloudformation-packaged.yaml
- aws s3 cp ./cloudformation-packaged.yaml <TARGET_S3_BUCKET>/cloudformation-packaged.yaml
In the result I get a deployment package and packaged cloudformation template in the TARGET_S3_BUCKET.
For each function in the ./src folder, I have a requirements.txt file that includes all the dependencies, but I dont run pip install -r requirements.txt manually.
Hey Lech! Thanks for the example. Given that this presumably only installs your source requirements (not Test Requirements - likerequests-mock,pytest, etc.), how do you run your tests during build? Does this setup allow you to use "packages that have natively compiled programs" (which usually requiredsam build --use-container)? Is there a reason why you're exporting thecloudformation-packaged.yamlfile to S3, rather than just referencing it directly in a "Deploy" phase of your CodePipeline?
– scubbo
Dec 31 '18 at 13:26
Hi, all good questions! :) 1. I dont do tests here, I only use CodeBuild for building. If you'd like to do tests, yes, I'd say you should run pip. 2. As for moving packaged file, potentially I guess you could do this with CodeDeploy, but since I have very specific and unusual structure of folders, I ended up doing it manually with AWS CLI :)
– Lech Migdal
Dec 31 '18 at 13:34
Got it, thanks! I'll wait to accept a more complete and/or generally applicable answer, then - but it was helpful to see someone else's setup, thank you!
– scubbo
Dec 31 '18 at 13:53
add a comment |
Please see below for buildspec.yaml that works for me when using AWS SAM with AWS CodeBuild, with cloudformation.yml
phases:
build:
commands:
- pip install --user aws-sam-cli
- USER_BASE_PATH=$(python -m site --user-base)
- export PATH=$PATH:$USER_BASE_PATH/bin
- sam build -t cloudformation.yml
- aws cloudformation package --template-file .aws-sam/build/template.yaml --s3-bucket <TARGET_S3_BUCKET> --output-template-file cloudformation-packaged.yaml
- aws s3 cp ./cloudformation-packaged.yaml <TARGET_S3_BUCKET>/cloudformation-packaged.yaml
In the result I get a deployment package and packaged cloudformation template in the TARGET_S3_BUCKET.
For each function in the ./src folder, I have a requirements.txt file that includes all the dependencies, but I dont run pip install -r requirements.txt manually.
Please see below for buildspec.yaml that works for me when using AWS SAM with AWS CodeBuild, with cloudformation.yml
phases:
build:
commands:
- pip install --user aws-sam-cli
- USER_BASE_PATH=$(python -m site --user-base)
- export PATH=$PATH:$USER_BASE_PATH/bin
- sam build -t cloudformation.yml
- aws cloudformation package --template-file .aws-sam/build/template.yaml --s3-bucket <TARGET_S3_BUCKET> --output-template-file cloudformation-packaged.yaml
- aws s3 cp ./cloudformation-packaged.yaml <TARGET_S3_BUCKET>/cloudformation-packaged.yaml
In the result I get a deployment package and packaged cloudformation template in the TARGET_S3_BUCKET.
For each function in the ./src folder, I have a requirements.txt file that includes all the dependencies, but I dont run pip install -r requirements.txt manually.
answered Dec 31 '18 at 13:11
Lech MigdalLech Migdal
1,59421735
1,59421735
Hey Lech! Thanks for the example. Given that this presumably only installs your source requirements (not Test Requirements - likerequests-mock,pytest, etc.), how do you run your tests during build? Does this setup allow you to use "packages that have natively compiled programs" (which usually requiredsam build --use-container)? Is there a reason why you're exporting thecloudformation-packaged.yamlfile to S3, rather than just referencing it directly in a "Deploy" phase of your CodePipeline?
– scubbo
Dec 31 '18 at 13:26
Hi, all good questions! :) 1. I dont do tests here, I only use CodeBuild for building. If you'd like to do tests, yes, I'd say you should run pip. 2. As for moving packaged file, potentially I guess you could do this with CodeDeploy, but since I have very specific and unusual structure of folders, I ended up doing it manually with AWS CLI :)
– Lech Migdal
Dec 31 '18 at 13:34
Got it, thanks! I'll wait to accept a more complete and/or generally applicable answer, then - but it was helpful to see someone else's setup, thank you!
– scubbo
Dec 31 '18 at 13:53
add a comment |
Hey Lech! Thanks for the example. Given that this presumably only installs your source requirements (not Test Requirements - likerequests-mock,pytest, etc.), how do you run your tests during build? Does this setup allow you to use "packages that have natively compiled programs" (which usually requiredsam build --use-container)? Is there a reason why you're exporting thecloudformation-packaged.yamlfile to S3, rather than just referencing it directly in a "Deploy" phase of your CodePipeline?
– scubbo
Dec 31 '18 at 13:26
Hi, all good questions! :) 1. I dont do tests here, I only use CodeBuild for building. If you'd like to do tests, yes, I'd say you should run pip. 2. As for moving packaged file, potentially I guess you could do this with CodeDeploy, but since I have very specific and unusual structure of folders, I ended up doing it manually with AWS CLI :)
– Lech Migdal
Dec 31 '18 at 13:34
Got it, thanks! I'll wait to accept a more complete and/or generally applicable answer, then - but it was helpful to see someone else's setup, thank you!
– scubbo
Dec 31 '18 at 13:53
Hey Lech! Thanks for the example. Given that this presumably only installs your source requirements (not Test Requirements - like
requests-mock, pytest, etc.), how do you run your tests during build? Does this setup allow you to use "packages that have natively compiled programs" (which usually required sam build --use-container)? Is there a reason why you're exporting the cloudformation-packaged.yaml file to S3, rather than just referencing it directly in a "Deploy" phase of your CodePipeline?– scubbo
Dec 31 '18 at 13:26
Hey Lech! Thanks for the example. Given that this presumably only installs your source requirements (not Test Requirements - like
requests-mock, pytest, etc.), how do you run your tests during build? Does this setup allow you to use "packages that have natively compiled programs" (which usually required sam build --use-container)? Is there a reason why you're exporting the cloudformation-packaged.yaml file to S3, rather than just referencing it directly in a "Deploy" phase of your CodePipeline?– scubbo
Dec 31 '18 at 13:26
Hi, all good questions! :) 1. I dont do tests here, I only use CodeBuild for building. If you'd like to do tests, yes, I'd say you should run pip. 2. As for moving packaged file, potentially I guess you could do this with CodeDeploy, but since I have very specific and unusual structure of folders, I ended up doing it manually with AWS CLI :)
– Lech Migdal
Dec 31 '18 at 13:34
Hi, all good questions! :) 1. I dont do tests here, I only use CodeBuild for building. If you'd like to do tests, yes, I'd say you should run pip. 2. As for moving packaged file, potentially I guess you could do this with CodeDeploy, but since I have very specific and unusual structure of folders, I ended up doing it manually with AWS CLI :)
– Lech Migdal
Dec 31 '18 at 13:34
Got it, thanks! I'll wait to accept a more complete and/or generally applicable answer, then - but it was helpful to see someone else's setup, thank you!
– scubbo
Dec 31 '18 at 13:53
Got it, thanks! I'll wait to accept a more complete and/or generally applicable answer, then - but it was helpful to see someone else's setup, thank you!
– scubbo
Dec 31 '18 at 13:53
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53987204%2fis-it-possible-recommended-to-use-sam-build-in-aws-codebuild%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown