Download as PDF, PPTX
![Call GitHub REST API with Guzzle
12
function gitHubAPI($uri, $token, $data = [], $method = 'GET')
{
$url = "https://api.github.com/$uri";
$headers = [
'Content-Type' => 'application/json',
'User-Agent' => 'my-org/my-app',
$headers['Authorization'] = "token " . $token();;
}
$guzzleParams = [ 'headers' => $headers, ];
if (!empty($data)) {
$method = 'POST';
$guzzleParams['json'] = $data;
}
$client = new GuzzleHttpClient();
$res = $client->request($method, $url, $guzzleParams);
$resultData = json_decode($res->getBody(), true);
$httpCode = $res->getStatusCode();
}](https://image.slidesharecdn.com/automateyourautomation-drupalconvienna-170926182503/75/Automate-Your-Automation-DrupalCon-Vienna-12-2048.jpg)
![Create Pull Request via REST API
13
https://developer.github.com/v3/pulls/#create-a-pull-request
gitHubAPI(
"/repos/$org/$repo/pulls",
$token,
[
"Title" => "Amazing new feature",
"Body" => "Please pull this in!",
"Head" => "$forkedRepo:new-feature",
"Base" => "master"
],
'POST'
);](https://image.slidesharecdn.com/automateyourautomation-drupalconvienna-170926182503/75/Automate-Your-Automation-DrupalCon-Vienna-13-2048.jpg)
![Add a Comment via REST API
15
https://developer.github.com/v3/issues/comments/#create-a-comment
gitHubAPI(
"/repos/$org/$repo/commits/$sha/comments",
$token,
[
"Body" => "Me too",
],
'POST'
);](https://image.slidesharecdn.com/automateyourautomation-drupalconvienna-170926182503/75/Automate-Your-Automation-DrupalCon-Vienna-15-2048.jpg)
![Automate setting up releases
34
Use the travis tool to automatically edit .travis.yml to include a release section:
$ travis setup releases --token=$GITHUB_TOKEN
Detected repository as org/my-tool, is this correct? |yes|
Username: greg-1-anderson
Password for greg-1-anderson: *************
Two-factor authentication code for greg-1-anderson:[REDACTED]
File to Upload: my-tool.phar
Deploy only from org/my-tool? |yes|
Encrypt API key? |yes|
If a token is provided, then the credentials prompt will be omitted.](https://image.slidesharecdn.com/automateyourautomation-drupalconvienna-170926182503/75/Automate-Your-Automation-DrupalCon-Vienna-34-2048.jpg)
![35
Self-Updating Phars
Robo PHP Task
Runner
Robo as a framework: https://robo.li/framework
Instructions on using Robo to create a standalone phar; adds
self:update command automatically.
$commandClasses = [ MyProjectCommandsRoboFile::class ];
$statusCode = RoboRobo::run(
$_SERVER['argv'],
$commandClasses,
'MyAppName',
'0.0.0-alpha0',
$output,
'my-org/my-project'
);
exit($statusCode);](https://image.slidesharecdn.com/automateyourautomation-drupalconvienna-170926182503/75/Automate-Your-Automation-DrupalCon-Vienna-35-2048.jpg)
![38
Automated Dependency Update Process
Travis
Cron
Composer
Lock Update
GitHub
Pull Request
Travis
Test Run
Merge
Pull Request
Exec hub REST REST
[1] [2]
1. Pull request is only created if there are updates available.
2. Pull request is only merged if the test passes.](https://image.slidesharecdn.com/automateyourautomation-drupalconvienna-170926182503/75/Automate-Your-Automation-DrupalCon-Vienna-38-2048.jpg)
![42
Add GitHub Token to Travis Settings
$ travis env set GITHUB_TOKEN "[PASTE VALUE FROM GITHUB]"
Circle CI REST API URL](https://image.slidesharecdn.com/automateyourautomation-drupalconvienna-170926182503/75/Automate-Your-Automation-DrupalCon-Vienna-42-2048.jpg)
![44
Only Update Dependencies in Cron
after_success:
- travis_retry php vendor/bin/coveralls -v
- |
if [ -z "$DO_POST_BUILD_ACTIONS" ] ; then
return
fi
if [ "$TRAVIS_EVENT_TYPE" != "cron" ] ; then
echo "Not a cron job; exiting."
return
fi
if [ -z "$GITHUB_TOKEN" ]; then
echo "No GITHUB_TOKEN defined; exiting."
return
fi
# … execution continues below](https://image.slidesharecdn.com/automateyourautomation-drupalconvienna-170926182503/75/Automate-Your-Automation-DrupalCon-Vienna-44-2048.jpg)
![45
Install and Run Composer Lock Updater
# … execution continues here from "after_success:" example
export PATH="$HOME/.composer/vendor/bin:$PATH"
composer global require danielbachhuber/composer-lock-updater
mkdir -p $HOME/bin
wget -O $HOME/bin/security-checker.phar [URL]
chmod +x $HOME/bin/security-checker.phar
export PATH="$HOME/bin:$PATH"
wget -O hub.tgz [URL]
tar -zxvf hub.tgz
export PATH=$PATH:$PWD/hub-linux-amd64-2.2.9/bin/
clu
For [URL], see full instructions in: https://github.com/danielbachhuber/composer-lock-updater](https://image.slidesharecdn.com/automateyourautomation-drupalconvienna-170926182503/75/Automate-Your-Automation-DrupalCon-Vienna-45-2048.jpg)
![48
Automatic Merge after Tests Pass
remote=$(git config --get remote.origin.url)
git remote set-url origin "${remote/github.com/$GITHUB_TOKEN:x-oauth-basic@github.com}"
if [ -n "$TRAVIS_COMMIT_RANGE" ] && [ "master" == "$TRAVIS_BRANCH" ]; then
changed_files=$(git diff --name-status "$TRAVIS_COMMIT_RANGE" | tr 't' ' ')
if [[ "$changed_files" == "M composer.lock" ]] ; then
(
echo "Only composer.lock was modified: auto-merging to $TRAVIS_BRANCH in $remote."
git reset "$TRAVIS_BRANCH"
git stash
git checkout "$TRAVIS_BRANCH"
git stash pop
git add composer.lock
git commit -m "Auto-update dependencies."
git push origin "$TRAVIS_BRANCH"
) 2>&1 | sed -e "s/$GITHUB_TOKEN/REDACTED/g"
else
echo "Not auto-merging, because multiple files were changed"
fi
fi](https://image.slidesharecdn.com/automateyourautomation-drupalconvienna-170926182503/75/Automate-Your-Automation-DrupalCon-Vienna-48-2048.jpg)
Uploaded byPantheon
Automate Your Automation | DrupalCon Vienna
Greg Anderson provides guidance on automating various development and deployment tasks. He discusses automating tasks like development, testing, deployment and maintenance. Some key tools mentioned are Travis CI, Circle CI, Composer and hub. Automating tasks improves reliability, makes onboarding easier and allows doing more work. The costs of not automating include increased risk of errors and lost knowledge over time.
More Related Content
Similar to Automate Your Automation | DrupalCon Vienna
Automate Your Automation | DrupalCon Vienna
- 1. Practical DevOps Greg Anderson AutomateYour Automation
- 2. 2 Greg Anderson Platform +Open Source Engineer Contributor to several popular OSS Projects: ● Drush ● Robo ● Consolidation Tool ● CGR greg_1_anderson
- 3. Automation Planning ● Identify somethingto Automate ● Make it good ● Survey available tools ● Automate! ● Repeat ● 3
- 4. What Can Be Automated? ●Development tasks ● Testing ● Deployment ● Maintenance (updates) 4
- 5. Investment and Benefit ● Morereliable and repeatable ● Easier to bring on new people ● Improvements compound ● It becomes possible to do more 5 https://xkcd.com/1205/
- 6. Cost of Not Automating ●Prone to errors and mistake ● Risk of lost knowledge ● Deferred maintenance piles up ● Discouragement 6
- 7. Extra Art 7 Tools ofthe Trade
- 8. Service APIs ● Createpull requests ● Post comments ● Configure test credentials ● Web hooks 8
- 9. Create Pull Requests 1. Pushbranch to GitHub 2. Create the PR ○ Use hub tool ○ Use REST API ● 9
- 10. 10 Push Branch toGitHub Pushing to a remote, and authenticated via ssh (e.g. from command line): $ git push -u origin my-pr-branch From a script with no remote set, and using an OAuth token: $ push https://$GITHUB_TOKEN:[email protected]/repo.git master
- 11. Install and Usehub CLI Tool 11 Create a pull request: $ hub pull-request -m "This is my awesome pull request." MacOS brew install hub choco install hub sudo apt-get install git-hub https://github.com/github/hub
- 12. Call GitHub RESTAPI with Guzzle 12 function gitHubAPI($uri, $token, $data = [], $method = 'GET') { $url = "https://api.github.com/$uri"; $headers = [ 'Content-Type' => 'application/json', 'User-Agent' => 'my-org/my-app', $headers['Authorization'] = "token " . $token();; } $guzzleParams = [ 'headers' => $headers, ]; if (!empty($data)) { $method = 'POST'; $guzzleParams['json'] = $data; } $client = new GuzzleHttpClient(); $res = $client->request($method, $url, $guzzleParams); $resultData = json_decode($res->getBody(), true); $httpCode = $res->getStatusCode(); }
- 13. Create Pull Requestvia REST API 13 https://developer.github.com/v3/pulls/#create-a-pull-request gitHubAPI( "/repos/$org/$repo/pulls", $token, [ "Title" => "Amazing new feature", "Body" => "Please pull this in!", "Head" => "$forkedRepo:new-feature", "Base" => "master" ], 'POST' );
- 14. Post Comments on PullRequests The hub tool does not support comments* , but the REST API is still available. 14 * Help improve hub! https://github.com/github/hub/pull/1465 Are you sure this will fly? Make sure the mock tests are passing before the demo. Has someone volunteered to do the cliff jump demo?
- 15. Add a Commentvia REST API 15 https://developer.github.com/v3/issues/comments/#create-a-comment gitHubAPI( "/repos/$org/$repo/commits/$sha/comments", $token, [ "Body" => "Me too", ], 'POST' );
- 16. Configure Test Credentials ● CircleCI ○ REST API ● Travis CI ○ CLI Tool ○ REST API 16
- 17. 17 Circle CI RESTAPI curl -X POST --header "Content-Type: application/json" -d '{"name":"foo", "value":"bar"}' https://circleci.com/api/v1.1/project/github/$user/$proj/envvar?circle-token=$t Obtain an OAuth token for circle-token at https://circleci.com/account/api. Important note: Circle uses github in its API URLs, even though it shortens this to gh in all of the URLs in its web application. https://circleci.com/docs/2.0/env-vars/#injecting-environment-variables-with-the-api
- 18. Travis CI CLITool 18 Set environment variables via env command: $ travis env set GITHUB_TOKEN $GITHUB_TOKEN For web API, see https://docs.travis-ci.com/api/#settings:-environment-variables https://github.com/travis-ci/travis.rb#env MacOS brew install travis Use the RubyInstaller sudo apt-get install ruby1.9.1-dev
- 19. Spinning Up New Projects ●Composer create-project ● Post-create project scripts 19
- 20. 20 Composer create-project ● Copiesand renames a repository and orphans / detatches the result. ● Updates from the source may still be pulled by adding a second remote: ○ git remote add upstream [email protected] :parent-org/parent-project.git ○ git pull upstream master ● Usually, parent is treated as a template project, and updates are not pulled. ● Things that will evolve over time are placed in dependencies. ● Dependencies are updated via composer update, as usual. https://github.com/drupal-composer/rupal-project
- 21. Post create-project scripts 21 ●Composer will run user-specified scripts after commands (e.g. install, update) ● This is used in drupal-composer/drupal-scaffold to download addition files. ● This allows files that cannot be stored in a dependency to be managed independently from the template repository. https://getcomposer.org/doc/articles/scripts.md
- 22.
- 23. Write Testable Code ● Passvalues to functions ● Return values from functions 23
- 24. Functional Testing ● Usesa working system to test ● Usually easy to write tests ● Sometimes hard to maintain (false failures creep in) ● Takes longer to run 24
- 25. Mocks ● Replaces anactual system ● Useful for providing values ● Can also collect results ● Sometimes hard to maintain (false passes can creep in) ● Best to avoid testing implementation, e.g. confirming order of API calls 25
- 26. Leveraging Docker ● Circle CI2.0 or Travis ● Maintain custom images ● Store system scripts in image ● Automatically rebuild your docker image whenever its source repository changes. 26
- 27. Parallelism in Circle2.0 27 https://circleci.com/docs/2.0/workflows/#using-workspaces-to-share-data-among-jobs defaults: &defaults docker: - image: quay.io/pantheon-public/build-tools-ci:1.x working_directory: ~/example_drops_8_composer environment: ADMIN_USERNAME: admin version: 2 jobs: build: <<: *defaults steps: - checkout - run: name: environment command: /build-tools-ci/scripts/set-environment
- 28. Create Custom DockerImage 28 DockerFile URL # Use an official Python runtime as a parent image FROM drupaldocker/php:7.1-cli # Set the working directory to /build-tools-ci WORKDIR /build-tools-ci # Copy the current directory into the container at /build-tools-ci ADD . /build-tools-ci # Collect the components we need for this image RUN apt-get update RUN composer -n global require -n "hirak/prestissimo:^0.3" RUN mkdir -p /usr/local/share/drush RUN /usr/bin/env COMPOSER_BIN_DIR=/usr/local/bin composer -n --working-dir=/usr/local/share/drush require drush/drush "^8"
- 29. Automate with Quayand GitHub 29
- 30.
- 31.
- 32.
- 33. Automating Releases ● Travis releasesteps ● Self Updates 33
- 34. Automate setting upreleases 34 Use the travis tool to automatically edit .travis.yml to include a release section: $ travis setup releases --token=$GITHUB_TOKEN Detected repository as org/my-tool, is this correct? |yes| Username: greg-1-anderson Password for greg-1-anderson: ************* Two-factor authentication code for greg-1-anderson:[REDACTED] File to Upload: my-tool.phar Deploy only from org/my-tool? |yes| Encrypt API key? |yes| If a token is provided, then the credentials prompt will be omitted.
- 35. 35 Self-Updating Phars Robo PHPTask Runner Robo as a framework: https://robo.li/framework Instructions on using Robo to create a standalone phar; adds self:update command automatically. $commandClasses = [ MyProjectCommandsRoboFile::class ]; $statusCode = RoboRobo::run( $_SERVER['argv'], $commandClasses, 'MyAppName', '0.0.0-alpha0', $output, 'my-org/my-project' ); exit($statusCode);
- 36.
- 37. Automating Composer Update ● Schedulevia Travis cron ● Composer Lock Updater ● Auto-merge after tests pass 37
- 38. 38 Automated Dependency UpdateProcess Travis Cron Composer Lock Update GitHub Pull Request Travis Test Run Merge Pull Request Exec hub REST REST [1] [2] 1. Pull request is only created if there are updates available. 2. Pull request is only merged if the test passes.
- 39. 39 Set Up TravisCI Cron Once configured, Travis schedules job to start right away:
- 40. 40 Add GitHub Tokento Travis Settings From https://github.com/settings/tokens: Circle CI REST API URL
- 41. 41 Specify Scopes Circle CIREST API URL
- 42. 42 Add GitHub Tokento Travis Settings $ travis env set GITHUB_TOKEN "[PASTE VALUE FROM GITHUB]" Circle CI REST API URL
- 43. 43 Prevent Redundant Execution Ifdoing highest / lowest testing with a matrix configuration entry, define an environment variable to control which Travis run will do the post-build actions: matrix: include: - php: 7.1 env: dependencies=highest env: DO_POST_BUILD_ACTIONS=1 - php: 7.0.11 - php: 5.6 - php: 5.5 env: dependencies=lowest
- 44. 44 Only Update Dependenciesin Cron after_success: - travis_retry php vendor/bin/coveralls -v - | if [ -z "$DO_POST_BUILD_ACTIONS" ] ; then return fi if [ "$TRAVIS_EVENT_TYPE" != "cron" ] ; then echo "Not a cron job; exiting." return fi if [ -z "$GITHUB_TOKEN" ]; then echo "No GITHUB_TOKEN defined; exiting." return fi # … execution continues below
- 45. 45 Install and RunComposer Lock Updater # … execution continues here from "after_success:" example export PATH="$HOME/.composer/vendor/bin:$PATH" composer global require danielbachhuber/composer-lock-updater mkdir -p $HOME/bin wget -O $HOME/bin/security-checker.phar [URL] chmod +x $HOME/bin/security-checker.phar export PATH="$HOME/bin:$PATH" wget -O hub.tgz [URL] tar -zxvf hub.tgz export PATH=$PATH:$PWD/hub-linux-amd64-2.2.9/bin/ clu For [URL], see full instructions in: https://github.com/danielbachhuber/composer-lock-updater
- 46. 46 Maintain Compatibility incomposer.lock The versions of dependencies returned for a given run of composer update may vary depending on what version of php you are running. For example, a lock file built with php 7.1 may include dependencies that do not work with php 5.6. Fix this with a "platform": "php" entry: "config": { "optimize-autoloader": true, "preferred-install": "dist", "sort-packages": true, "platform": { "php": "5.6" } },
- 47. 47 Update Details inCommit Comment PR is attributed to user who provided GitHub token. Comment lists dependencies that were updated. Security report included to bring attention to vulnerabilities. Commit itself is authored by composer-lock-updateuser.
- 48. 48 Automatic Merge afterTests Pass remote=$(git config --get remote.origin.url) git remote set-url origin "${remote/github.com/$GITHUB_TOKEN:[email protected]}" if [ -n "$TRAVIS_COMMIT_RANGE" ] && [ "master" == "$TRAVIS_BRANCH" ]; then changed_files=$(git diff --name-status "$TRAVIS_COMMIT_RANGE" | tr 't' ' ') if [[ "$changed_files" == "M composer.lock" ]] ; then ( echo "Only composer.lock was modified: auto-merging to $TRAVIS_BRANCH in $remote." git reset "$TRAVIS_BRANCH" git stash git checkout "$TRAVIS_BRANCH" git stash pop git add composer.lock git commit -m "Auto-update dependencies." git push origin "$TRAVIS_BRANCH" ) 2>&1 | sed -e "s/$GITHUB_TOKEN/REDACTED/g" else echo "Not auto-merging, because multiple files were changed" fi fi
- 49. Automate the automation ofyour automation Run a command to automatically configure Travis to automate the updating of your project’s composer.lock file. 49
- 50. 50 The ci travis:cluCommand ● Install ci.phar. ● Change your working directory to your PHP project. ● Define $GITHUB_TOKEN ● Run ci travis:clu ● Inspect the changes. Push them up to GitHub. ● Turn on cron in the Travis admin interface ● Be amazed.
- 51. Extra Art 51 Finished! NotFinished!
- 52. JOIN US FOR CONTRIBUTIONSPRINT Friday, 29 September, 2017 First time Sprinter Workshop Mentored Core Sprint General Sprint 9:00-12:00 Room: Lehar 1 - Lehar 2 9:00-18:00 Room: Stolz 2 9:00-18:00 Room: Mall #drupalsprints
- 53. WHAT DID YOUTHINK? Locate this session at the DrupalCon Vienna website: http://vienna2017.drupal.org/schedule Take the survey! https://www.surveymonkey.com/r/drupalconvienna