Skip to content
Application Development II
GitLabGitHub

CI/CD with GitHub

How to develop a project of any size in continous delivery and deployment using modern GitHub techniques
PreviousPDFSlidesNext
Overview

Ran out of time to write this properly, but you should know the following things (loosely based on explanations I have given in-class)

  • git is the distributed repository technogy. It is a phenomenally powerful and ubiquitious open-source tool used for software collaboration for decades.
  • GitHub is a relatively modern/recent project that is essentially a web service that manages git servers for its clients.
  • Before GitHub, people ran their own git servers! Or, better yet, people emailed git commit patches to update eachothers codebases. I’m not joking, emailing git commit patches remains the recommended method to contribute to the Linux Kernel to this day, and it actually makes a ton of sense.
  • Actually, the word distributed means something here: every local git repository can act as a remote git repository. Having a server that listens to HTTPS/SSH calls on git commands is all that elevates a local repository to a remote git “server”
  • After git but before GitHub, there were a wide variety of alternative technologies for this kind of thing. You may hear about SVN/Mercurial, that’s what they are. They are still in use (I used Mercurial at work for a job in 2012) but I rarely run into them anymore. Similar idea but different.
  • Today, there are many alternatives to using GitHub for hosting your source repositories and managing things to do with that source code. We used it this semester out of convenience/familiarity, but you should know there are other options:
    • self hosting a git server as discussed above (if you want to learn about self hosting, this is a great project!)
    • similar SAAS to Github like GitLab
    • commercial software like Bitbucket
    • A variety of things that are somewhat similar to GitHub in what they provide. I would personally recommend checking out sourcehut, Codeberg, and foregjo as my understanding of the most sophisticated, instructive, useful, and “free as in libre” source software distribution platforms.

Github makes use of Pull Requests and Actions to implement a CI/CD workflow, elaborated on in the sections below.

Pull request templates are .github directory Markdown files that allow you to specify a template for Pull Requests. You may already be using one for your project milestone.

When you add a pull request template to your repository, project contributors will automatically see the template’s contents in the pull request body.

Note: templates must be created on the repository’s default branch. Templates created in other branches are not available for collaborators to use. Pull request template filenames are not case sensitive, and can have an extension such as .md or .txt.

Pull requests give developer teams they give us the opportunity to review, then discard/rework/edit, proposed changes to branch history in a shared draft environment. Pull requests are essentially a draft of the future history (whoa!) of your main branch. More clearly, they are requests to pull (merge/rebase) the commit history of one branch onto another, and most typically are used to pull developer changes onto a shared branch (typically called main).

In GitHub, Pull Requests (PRs) are based on a pulling a source branch onto a target branch. They are updated automatically when the source branch they are based on is updated (i.e., when changes pushed are pushed).

This is all review so far — however, there are slightly different workflows depending on whether or not the changes that are pushed have had their history altered or produce change conflicts.

It is impossible to git push a branch that has history/content-conflicts — those conflicts have to be “merged” — but then, what happens to the nice linear history you spent time creating? In those cases, we can use git push --force to overwrite the remote developer branch with your proposed changes, thereby updating the pull request without any fuss.

You can incrementally update a pull request by simply git push when you have new commits. As long as the new commits are linear additions to the existing pull request branch, and no modification of branch history took place, the pull request can update automatically along with the branch update.

Terminal window
## Ensure you are on your developer branch,
## and ensure the upstream branch you are pushing to is your developer branch
git push
Terminal window
## Ensure you are on your developer branch,
## and ensure the upstream branch you are pushing to is your developer branch
git push

Figure: GitHub pull request, linear set of commits

You will see that the pull request automatically updates when the branch is pushed.

After making a series of changes, you may find that your pull request has conflicts with the main branch that cannot be automatically resolved:

You will need to update your pull request such that it has the latest code from main (git pull --rebase) and resolve the conflicts on your local git. This will likely involve rebasing your developer branch history so that you can still retain a linear series of commits. See the relevant sections of the git basics and git advanced for more information.

It is more complicated to push a branch (and therefore update a pull request) when you have done “destructive” operations like git rebase — the unique ID of historical commits, and relationships between them, are destroyed when github history is altered, therefore the pushed branch is no longer compatible.

You will likely see screens that look something like this before and after your branch rebase:

$ git status
On branch nav-codelab
Your branch and 'origin/nav-codelab' have diverged,
and have 2 and 2 different commits each, respectively.
  (use "git pull" if you want to integrate the remote branch with yours)


nothing to commit, working tree clean


$ git push
Username for 'https://github.com': ...
Password for 'https://michaelhaaf@github.com': ...
To https://github.com/michaelhaaf/5A6-F23-assignment3
 ! [rejected]        nav-codelab -> nav-codelab (non-fast-forward)  ### non-fast-forward is the reason for rejection
error: failed to push some refs to 'https://github.com/michaelhaaf/5A6-F23-assignment3'
hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart. If you want to integrate the remote changes,
hint: use 'git pull' before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
$ git status
On branch nav-codelab
Your branch and 'origin/nav-codelab' have diverged,
and have 2 and 2 different commits each, respectively.
  (use "git pull" if you want to integrate the remote branch with yours)


nothing to commit, working tree clean


$ git push
Username for 'https://github.com': ...
Password for 'https://michaelhaaf@github.com': ...
To https://github.com/michaelhaaf/5A6-F23-assignment3
 ! [rejected]        nav-codelab -> nav-codelab (non-fast-forward)  ### non-fast-forward is the reason for rejection
error: failed to push some refs to 'https://github.com/michaelhaaf/5A6-F23-assignment3'
hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart. If you want to integrate the remote changes,
hint: use 'git pull' before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

How can we update the pull request?

Note well: it is not necessary to create a new branch, nor to create a new pull request, when the local git history has changed! Updating a developer branch that is under review is the perfect use case for a “dangerous” option like git push --force, which simply overrites the remote branch.

Terminal window
## Be careful to make sure you are on the right branch!
git push --force
Terminal window
## Be careful to make sure you are on the right branch!
git push --force

Figure: GitHub pull request, no issues, “@user force-pushed SHA-1 onto SHA-2” or similar and history is updated.

This is why using developer branches for code changes so useful: being able to manipulate your draft work history on the fly is essential for collaborating on code with other developers, in order to address feedback — and are options are much more limited if we develop on main directly for that reason.

Workflows are, in a nutshell, a way to get the GitHub Server virtual machine that your code is run on to execute a variety of actions related to your codebase. For Application Development, this typically means you can write instructions to prepare dependencies, build your app, run tests, deploy your app, etc. Workflows are actually incredibly powerful (basically an interface for you to run scripts with a variety of useful libraries on free servers — you can actually do all kind of neat stuff beyond app development with this! Food for thought).

Basically:

  • workflow is a set of instructions written in a file (typically .yml)
  • each workflow has a set of triggers that launch the workflow (manual with workflow-dispatch or based on repository events)
  • each workflow has a set of jobs it runs
  • each job has a set of actions it can run
  • actions can be written and shared (see Action Marketplace) so that you don’t have to re-write common scripts
  • actions can also just be custom scripts (bash, like chmod +x ./gradlew, but other shells are possible)

More information:

Actions written by other developers are maintained on the GitHub Marketplace — despite what the name implies, most of these are free to use! We have been using actions like actions/checkout and actions/setup-java throughout the course. In Milestone 4b, I’ve provided some recommended actions to use, but feel free to look around this list to see if there are other useful ones (and, if you see the workflow of another developer, you’ll know where these uses statements come from and where to find out what they do.)

This section explains an example GitHub Action that’s particularly useful for a Milestone 4b requirement. You should have already set up a Firebase account and basic Firebase dependencies/authentication before starting this one (review Talib’s notes for explanations of how to do this.)

My notes here are a bit threadbare — I have a few screenshots from key stages along the way that may be helpful.

I implemented this workflow based on the following tutorials/instructions:

In case you’re having trouble finding where to go in the tutorials above:

Finding the Project settings button (gearbox top left of Firebase console)

The images below show what your GitHub secret setup should look like. You will have access to variables in your GitHub Workflows of the form secrets.VARIABLE_NAME. This is explained in more detail in the tutorial, the M4b Issue I created, and the github documentation about secrets in github actions.

(this basically lets you upload encrypted data to GitHub and safely giveing the GitHub Actions virtual machine the ability to use/decrypt that data.) (Advanced: a similar technique is used to automatically sign .apk files, which is a prerequisite for uploading your apps to Google Play automatically.)

Dump the contents of a json file into the “Secret” box. Give the secret an all-caps meaningful “Name”.

These notes are sparse but there’s some good documentation here organized by section for reference.

  • Requirement: used to define high-level project requirements that are broken up into tasks using Work Breakdown Structure WBS
  • Task: used to define Requirement subtasks using Level of Effort (LOE) estimations

Roadmaps are relatively common features in open-source projects for keeping-in-the-loop your community of developers, contributers, and users of your software — the “roadmap” project is a recent attempt by GitHub to provide a GUI for automating some of this effort.

To adapt your proposal to a GitHub roadmap, see the links below for documentation and tutorials:

These articles probably belong in another lecture, but I’ve run out of time. Check them out!