Is it possible/recommended to use `sam build` in AWS CodeBuild?












0















This question spun out of this one. Now that I have a better understanding of what was going wrong there, and a workable, if imperfect, solution, I'm submitting a more focused follow-up (I'm still something of a novice at StackOverflow - please let me know if this contravenes etiquette, and I should follow-up on the original).



This page suggests that "You use AWS CodeBuild to build, locally test, and package your serverless application". However, when I include a sam build command in my buildspec.yml, I get the following log output, suggesting that sam is not installed on CodeBuild images:



[Container] 2018/12/31 11:41:49 Running command sam build --use-container 
sh: 1: sam: not found

[Container] 2018/12/31 11:41:49 Command did not exit successfully sam build --use-container exit status 127
[Container] 2018/12/31 11:41:49 Phase complete: BUILD Success: false
[Container] 2018/12/31 11:41:49 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: sam build --use-container. Reason: exit status 127


Furthermore, if I install SAM with pip install aws-sam-cli, running sam build --use-container in CodeBuild gives an error. sam build itself succeeds, but since it doesn't install test dependencies, I'd still need to use pip install -r tests/requirements-test.txt -t . to be able to run tests in CodeBuild. Moreover, this suggests that --use-container is required for "packages that have natively compiled programs").



This makes me wonder whether I'm trying to do something wrong. What's the recommended way of building SAM services in a CI/CD workflow on AWS?










share|improve this question





























    0















    This question spun out of this one. Now that I have a better understanding of what was going wrong there, and a workable, if imperfect, solution, I'm submitting a more focused follow-up (I'm still something of a novice at StackOverflow - please let me know if this contravenes etiquette, and I should follow-up on the original).



    This page suggests that "You use AWS CodeBuild to build, locally test, and package your serverless application". However, when I include a sam build command in my buildspec.yml, I get the following log output, suggesting that sam is not installed on CodeBuild images:



    [Container] 2018/12/31 11:41:49 Running command sam build --use-container 
    sh: 1: sam: not found

    [Container] 2018/12/31 11:41:49 Command did not exit successfully sam build --use-container exit status 127
    [Container] 2018/12/31 11:41:49 Phase complete: BUILD Success: false
    [Container] 2018/12/31 11:41:49 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: sam build --use-container. Reason: exit status 127


    Furthermore, if I install SAM with pip install aws-sam-cli, running sam build --use-container in CodeBuild gives an error. sam build itself succeeds, but since it doesn't install test dependencies, I'd still need to use pip install -r tests/requirements-test.txt -t . to be able to run tests in CodeBuild. Moreover, this suggests that --use-container is required for "packages that have natively compiled programs").



    This makes me wonder whether I'm trying to do something wrong. What's the recommended way of building SAM services in a CI/CD workflow on AWS?










    share|improve this question



























      0












      0








      0








      This question spun out of this one. Now that I have a better understanding of what was going wrong there, and a workable, if imperfect, solution, I'm submitting a more focused follow-up (I'm still something of a novice at StackOverflow - please let me know if this contravenes etiquette, and I should follow-up on the original).



      This page suggests that "You use AWS CodeBuild to build, locally test, and package your serverless application". However, when I include a sam build command in my buildspec.yml, I get the following log output, suggesting that sam is not installed on CodeBuild images:



      [Container] 2018/12/31 11:41:49 Running command sam build --use-container 
      sh: 1: sam: not found

      [Container] 2018/12/31 11:41:49 Command did not exit successfully sam build --use-container exit status 127
      [Container] 2018/12/31 11:41:49 Phase complete: BUILD Success: false
      [Container] 2018/12/31 11:41:49 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: sam build --use-container. Reason: exit status 127


      Furthermore, if I install SAM with pip install aws-sam-cli, running sam build --use-container in CodeBuild gives an error. sam build itself succeeds, but since it doesn't install test dependencies, I'd still need to use pip install -r tests/requirements-test.txt -t . to be able to run tests in CodeBuild. Moreover, this suggests that --use-container is required for "packages that have natively compiled programs").



      This makes me wonder whether I'm trying to do something wrong. What's the recommended way of building SAM services in a CI/CD workflow on AWS?










      share|improve this question
















      This question spun out of this one. Now that I have a better understanding of what was going wrong there, and a workable, if imperfect, solution, I'm submitting a more focused follow-up (I'm still something of a novice at StackOverflow - please let me know if this contravenes etiquette, and I should follow-up on the original).



      This page suggests that "You use AWS CodeBuild to build, locally test, and package your serverless application". However, when I include a sam build command in my buildspec.yml, I get the following log output, suggesting that sam is not installed on CodeBuild images:



      [Container] 2018/12/31 11:41:49 Running command sam build --use-container 
      sh: 1: sam: not found

      [Container] 2018/12/31 11:41:49 Command did not exit successfully sam build --use-container exit status 127
      [Container] 2018/12/31 11:41:49 Phase complete: BUILD Success: false
      [Container] 2018/12/31 11:41:49 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: sam build --use-container. Reason: exit status 127


      Furthermore, if I install SAM with pip install aws-sam-cli, running sam build --use-container in CodeBuild gives an error. sam build itself succeeds, but since it doesn't install test dependencies, I'd still need to use pip install -r tests/requirements-test.txt -t . to be able to run tests in CodeBuild. Moreover, this suggests that --use-container is required for "packages that have natively compiled programs").



      This makes me wonder whether I'm trying to do something wrong. What's the recommended way of building SAM services in a CI/CD workflow on AWS?







      amazon-web-services aws-codebuild aws-serverless






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Dec 31 '18 at 12:12







      scubbo

















      asked Dec 31 '18 at 11:55









      scubboscubbo

      1,18642139




      1,18642139
























          1 Answer
          1






          active

          oldest

          votes


















          0














          Please see below for buildspec.yaml that works for me when using AWS SAM with AWS CodeBuild, with cloudformation.yml



          phases:
          build:
          commands:
          - pip install --user aws-sam-cli
          - USER_BASE_PATH=$(python -m site --user-base)
          - export PATH=$PATH:$USER_BASE_PATH/bin
          - sam build -t cloudformation.yml
          - aws cloudformation package --template-file .aws-sam/build/template.yaml --s3-bucket <TARGET_S3_BUCKET> --output-template-file cloudformation-packaged.yaml
          - aws s3 cp ./cloudformation-packaged.yaml <TARGET_S3_BUCKET>/cloudformation-packaged.yaml


          In the result I get a deployment package and packaged cloudformation template in the TARGET_S3_BUCKET.



          For each function in the ./src folder, I have a requirements.txt file that includes all the dependencies, but I dont run pip install -r requirements.txt manually.






          share|improve this answer
























          • Hey Lech! Thanks for the example. Given that this presumably only installs your source requirements (not Test Requirements - like requests-mock, pytest, etc.), how do you run your tests during build? Does this setup allow you to use "packages that have natively compiled programs" (which usually required sam build --use-container)? Is there a reason why you're exporting the cloudformation-packaged.yaml file to S3, rather than just referencing it directly in a "Deploy" phase of your CodePipeline?

            – scubbo
            Dec 31 '18 at 13:26











          • Hi, all good questions! :) 1. I dont do tests here, I only use CodeBuild for building. If you'd like to do tests, yes, I'd say you should run pip. 2. As for moving packaged file, potentially I guess you could do this with CodeDeploy, but since I have very specific and unusual structure of folders, I ended up doing it manually with AWS CLI :)

            – Lech Migdal
            Dec 31 '18 at 13:34











          • Got it, thanks! I'll wait to accept a more complete and/or generally applicable answer, then - but it was helpful to see someone else's setup, thank you!

            – scubbo
            Dec 31 '18 at 13:53











          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53987204%2fis-it-possible-recommended-to-use-sam-build-in-aws-codebuild%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          0














          Please see below for buildspec.yaml that works for me when using AWS SAM with AWS CodeBuild, with cloudformation.yml



          phases:
          build:
          commands:
          - pip install --user aws-sam-cli
          - USER_BASE_PATH=$(python -m site --user-base)
          - export PATH=$PATH:$USER_BASE_PATH/bin
          - sam build -t cloudformation.yml
          - aws cloudformation package --template-file .aws-sam/build/template.yaml --s3-bucket <TARGET_S3_BUCKET> --output-template-file cloudformation-packaged.yaml
          - aws s3 cp ./cloudformation-packaged.yaml <TARGET_S3_BUCKET>/cloudformation-packaged.yaml


          In the result I get a deployment package and packaged cloudformation template in the TARGET_S3_BUCKET.



          For each function in the ./src folder, I have a requirements.txt file that includes all the dependencies, but I dont run pip install -r requirements.txt manually.






          share|improve this answer
























          • Hey Lech! Thanks for the example. Given that this presumably only installs your source requirements (not Test Requirements - like requests-mock, pytest, etc.), how do you run your tests during build? Does this setup allow you to use "packages that have natively compiled programs" (which usually required sam build --use-container)? Is there a reason why you're exporting the cloudformation-packaged.yaml file to S3, rather than just referencing it directly in a "Deploy" phase of your CodePipeline?

            – scubbo
            Dec 31 '18 at 13:26











          • Hi, all good questions! :) 1. I dont do tests here, I only use CodeBuild for building. If you'd like to do tests, yes, I'd say you should run pip. 2. As for moving packaged file, potentially I guess you could do this with CodeDeploy, but since I have very specific and unusual structure of folders, I ended up doing it manually with AWS CLI :)

            – Lech Migdal
            Dec 31 '18 at 13:34











          • Got it, thanks! I'll wait to accept a more complete and/or generally applicable answer, then - but it was helpful to see someone else's setup, thank you!

            – scubbo
            Dec 31 '18 at 13:53
















          0














          Please see below for buildspec.yaml that works for me when using AWS SAM with AWS CodeBuild, with cloudformation.yml



          phases:
          build:
          commands:
          - pip install --user aws-sam-cli
          - USER_BASE_PATH=$(python -m site --user-base)
          - export PATH=$PATH:$USER_BASE_PATH/bin
          - sam build -t cloudformation.yml
          - aws cloudformation package --template-file .aws-sam/build/template.yaml --s3-bucket <TARGET_S3_BUCKET> --output-template-file cloudformation-packaged.yaml
          - aws s3 cp ./cloudformation-packaged.yaml <TARGET_S3_BUCKET>/cloudformation-packaged.yaml


          In the result I get a deployment package and packaged cloudformation template in the TARGET_S3_BUCKET.



          For each function in the ./src folder, I have a requirements.txt file that includes all the dependencies, but I dont run pip install -r requirements.txt manually.






          share|improve this answer
























          • Hey Lech! Thanks for the example. Given that this presumably only installs your source requirements (not Test Requirements - like requests-mock, pytest, etc.), how do you run your tests during build? Does this setup allow you to use "packages that have natively compiled programs" (which usually required sam build --use-container)? Is there a reason why you're exporting the cloudformation-packaged.yaml file to S3, rather than just referencing it directly in a "Deploy" phase of your CodePipeline?

            – scubbo
            Dec 31 '18 at 13:26











          • Hi, all good questions! :) 1. I dont do tests here, I only use CodeBuild for building. If you'd like to do tests, yes, I'd say you should run pip. 2. As for moving packaged file, potentially I guess you could do this with CodeDeploy, but since I have very specific and unusual structure of folders, I ended up doing it manually with AWS CLI :)

            – Lech Migdal
            Dec 31 '18 at 13:34











          • Got it, thanks! I'll wait to accept a more complete and/or generally applicable answer, then - but it was helpful to see someone else's setup, thank you!

            – scubbo
            Dec 31 '18 at 13:53














          0












          0








          0







          Please see below for buildspec.yaml that works for me when using AWS SAM with AWS CodeBuild, with cloudformation.yml



          phases:
          build:
          commands:
          - pip install --user aws-sam-cli
          - USER_BASE_PATH=$(python -m site --user-base)
          - export PATH=$PATH:$USER_BASE_PATH/bin
          - sam build -t cloudformation.yml
          - aws cloudformation package --template-file .aws-sam/build/template.yaml --s3-bucket <TARGET_S3_BUCKET> --output-template-file cloudformation-packaged.yaml
          - aws s3 cp ./cloudformation-packaged.yaml <TARGET_S3_BUCKET>/cloudformation-packaged.yaml


          In the result I get a deployment package and packaged cloudformation template in the TARGET_S3_BUCKET.



          For each function in the ./src folder, I have a requirements.txt file that includes all the dependencies, but I dont run pip install -r requirements.txt manually.






          share|improve this answer













          Please see below for buildspec.yaml that works for me when using AWS SAM with AWS CodeBuild, with cloudformation.yml



          phases:
          build:
          commands:
          - pip install --user aws-sam-cli
          - USER_BASE_PATH=$(python -m site --user-base)
          - export PATH=$PATH:$USER_BASE_PATH/bin
          - sam build -t cloudformation.yml
          - aws cloudformation package --template-file .aws-sam/build/template.yaml --s3-bucket <TARGET_S3_BUCKET> --output-template-file cloudformation-packaged.yaml
          - aws s3 cp ./cloudformation-packaged.yaml <TARGET_S3_BUCKET>/cloudformation-packaged.yaml


          In the result I get a deployment package and packaged cloudformation template in the TARGET_S3_BUCKET.



          For each function in the ./src folder, I have a requirements.txt file that includes all the dependencies, but I dont run pip install -r requirements.txt manually.







          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered Dec 31 '18 at 13:11









          Lech MigdalLech Migdal

          1,59421735




          1,59421735













          • Hey Lech! Thanks for the example. Given that this presumably only installs your source requirements (not Test Requirements - like requests-mock, pytest, etc.), how do you run your tests during build? Does this setup allow you to use "packages that have natively compiled programs" (which usually required sam build --use-container)? Is there a reason why you're exporting the cloudformation-packaged.yaml file to S3, rather than just referencing it directly in a "Deploy" phase of your CodePipeline?

            – scubbo
            Dec 31 '18 at 13:26











          • Hi, all good questions! :) 1. I dont do tests here, I only use CodeBuild for building. If you'd like to do tests, yes, I'd say you should run pip. 2. As for moving packaged file, potentially I guess you could do this with CodeDeploy, but since I have very specific and unusual structure of folders, I ended up doing it manually with AWS CLI :)

            – Lech Migdal
            Dec 31 '18 at 13:34











          • Got it, thanks! I'll wait to accept a more complete and/or generally applicable answer, then - but it was helpful to see someone else's setup, thank you!

            – scubbo
            Dec 31 '18 at 13:53



















          • Hey Lech! Thanks for the example. Given that this presumably only installs your source requirements (not Test Requirements - like requests-mock, pytest, etc.), how do you run your tests during build? Does this setup allow you to use "packages that have natively compiled programs" (which usually required sam build --use-container)? Is there a reason why you're exporting the cloudformation-packaged.yaml file to S3, rather than just referencing it directly in a "Deploy" phase of your CodePipeline?

            – scubbo
            Dec 31 '18 at 13:26











          • Hi, all good questions! :) 1. I dont do tests here, I only use CodeBuild for building. If you'd like to do tests, yes, I'd say you should run pip. 2. As for moving packaged file, potentially I guess you could do this with CodeDeploy, but since I have very specific and unusual structure of folders, I ended up doing it manually with AWS CLI :)

            – Lech Migdal
            Dec 31 '18 at 13:34











          • Got it, thanks! I'll wait to accept a more complete and/or generally applicable answer, then - but it was helpful to see someone else's setup, thank you!

            – scubbo
            Dec 31 '18 at 13:53

















          Hey Lech! Thanks for the example. Given that this presumably only installs your source requirements (not Test Requirements - like requests-mock, pytest, etc.), how do you run your tests during build? Does this setup allow you to use "packages that have natively compiled programs" (which usually required sam build --use-container)? Is there a reason why you're exporting the cloudformation-packaged.yaml file to S3, rather than just referencing it directly in a "Deploy" phase of your CodePipeline?

          – scubbo
          Dec 31 '18 at 13:26





          Hey Lech! Thanks for the example. Given that this presumably only installs your source requirements (not Test Requirements - like requests-mock, pytest, etc.), how do you run your tests during build? Does this setup allow you to use "packages that have natively compiled programs" (which usually required sam build --use-container)? Is there a reason why you're exporting the cloudformation-packaged.yaml file to S3, rather than just referencing it directly in a "Deploy" phase of your CodePipeline?

          – scubbo
          Dec 31 '18 at 13:26













          Hi, all good questions! :) 1. I dont do tests here, I only use CodeBuild for building. If you'd like to do tests, yes, I'd say you should run pip. 2. As for moving packaged file, potentially I guess you could do this with CodeDeploy, but since I have very specific and unusual structure of folders, I ended up doing it manually with AWS CLI :)

          – Lech Migdal
          Dec 31 '18 at 13:34





          Hi, all good questions! :) 1. I dont do tests here, I only use CodeBuild for building. If you'd like to do tests, yes, I'd say you should run pip. 2. As for moving packaged file, potentially I guess you could do this with CodeDeploy, but since I have very specific and unusual structure of folders, I ended up doing it manually with AWS CLI :)

          – Lech Migdal
          Dec 31 '18 at 13:34













          Got it, thanks! I'll wait to accept a more complete and/or generally applicable answer, then - but it was helpful to see someone else's setup, thank you!

          – scubbo
          Dec 31 '18 at 13:53





          Got it, thanks! I'll wait to accept a more complete and/or generally applicable answer, then - but it was helpful to see someone else's setup, thank you!

          – scubbo
          Dec 31 '18 at 13:53


















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53987204%2fis-it-possible-recommended-to-use-sam-build-in-aws-codebuild%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Mossoró

          Error while reading .h5 file using the rhdf5 package in R

          Pushsharp Apns notification error: 'InvalidToken'