Wikia

Community Leadership Summit Wiki

The OpenStack Continuous Integration Process and How It Shapes the Community

Comments0
178pages on
this wiki

Mark Atwood and Elizabeth Krumbach



OpenStack has 500-700 contributors per 30 day window, 200 contributing companies, 170  commits per day

every commit is fully tested in a 2.5 hour cycle



this process formed in response to earlier anti-patterns eg Java



how it works:

review.openstack.org

check subproj out of git

make your change

gitreview

(CLA must be signed somewhere in here, or you don't get an SSH key)

pushes the contribution to a special tree managed by robots, smoke tests are run automatically, results sent up to the metadata

Gerrit is a code review system bought by Google and rewritten in Java, works well with Git - event stream: ssh to a port, send JSON commands

(OpenStack's version may become a fork as Google is not taking their patches.)

anyone can do a +1 or -1 code review for tech nature, comments, git commit message well formatted? "no assholes" rule applies although nitpicking is allowed

first level reviewers eventually get social cred and may be recommended by the core reviewers - based on merit, not employer

one core reviewer (preferably 2) for each project give a +2, then the change moves into a more extensive test system (2.5 hours)

there are no bypasses, process is the same for all

"everybody and nobody" has the right to commit

tip always works - you can always check out a release and it will work

merges happen all the time 



git with gitreview plug-in

Zuul is the gatekeeper

with multiple patches, Zuul spins up open compute instances and test trunk + A, trunk + A + B, etc.

if all works, all get committed

if not, continue combination testing until successful combinations found - those are passed



the whole system is another OpenStack project, OpenStack Infra, which can also be changed via the same process



how many core reviewers? depends on the project - 12-18 in some

OpenStack is a loosely-coupled set of components speaking to each other via REST

very decentralized - has to be, due to the membership of the large OpenStack project



talking to MediaWiki and KDE about using this system to review



this system leads to continuous deployment



where do the tests come from?

added as part of dev - every feature includes test cases

there is a QA team that keeps tests under control so the gate process doesn't become too cumbersome



    • this is documented at ci.openstack.org **



use a work in progress tag for intermediate review, non-final code



there is a technical committee that decides eg what we're going to test on, therefore what will be in the gate



onboarding mechanisms? one recently had a hackathon / trainathon for the CI system

other projects do their own onboarding

the automation makes minor initial contribs easy so people can onboard themselves

there is a practice system that resets every 24 hours

"how to get involved" page on the wiki links to the Garrett page

you need to know git, and do gitreview rather than git pull to submit it to Garrett

newbies don't need to be aware of what's going on behind the scenes

if there's a problem, system logs will be sent

there are dashboards to see status, most changes are accepted within hours, also sends status emails for each step



LaunchPad for bugs, ID, and blueprints



much of this is lifted from the Ubuntu process



docs maintained as part of source code in git means you can reject a push which does not include docs - docs are done via git



if something is not touched in 7 days after a -1, it is marked as "abandoned", but can be resurrected if needed - this is just to keep it out of review queues if nothing is happening on it



review day every 2 weeks - see status.openstack.org - to clean up review lists



OpenStack uses github as a public mirror - pull requests not accepted directly from github

Monty Taylor architected this



"We have contributors from the NSA."



level 1 reviewers are not necessarily core contributors; level 2 reviewers typically are



OpenStack has many contributors who are paid to contribute as their day jobs

engagement of corp QA not so great, but they do contribute to Tempest which is a QA project



(UTest as a crowd source for testing - check it out)



How to set up your own? see the runbook - would take a skilled sysadmin half day of work BUT it does assume that you're running on an OpenStack system, could be difficult if you're not

OpenStack installation is not easy at all



adoption:

data localization is a growing issue with recent US NSA revelations -> local clouds will be popular

there is not a public OpenStack cloud in every jurisdiction yet, but there is growing impetus for it



  1. openstackinfra on freenode



parallelizable tests preferred, but WIP

assuming that computation is infinite and free



it does break sometimes, ie because of a new dependency

Around Wikia's network

Random Wiki