TL;DR - All the apps we build and test are linked from a single repo.

This is just a short update on progress with the CODE-RADE project with a description of where to find the code and how to contribute.

How testing works.

A huge aspect of CODE-RADE is modularity - we build and test nodes in a dependency tree, from base dependencies all the way up. Instead of providing a single binary result (yes, the repo is working, no there are problems), we provide a result for each node in the tree. Specifically, that means that every application is compiled against its dependencies and tested before the next node in the tree is done.

A second aspect of CODE-RADE is continuous integration - tests would run at every commit of code to a repository. Modularity implied that only the node that has changed should be tested, and that this test should trigger all of the upstream tests in order to see if there were regression problems. The alternative would be to have monolithic testing where the full tree was traversed on a predetermined schedule, taking into account any changes that had been made.

While these two approaches are not necessarily contradictory, the first had the agility which we felt was one of the strong points of CODE-RADE for community adoption, and the benefit of giving fairly fast responses ¹ to any changes made by a contributor.

Originally, we had the idea that everything would go into a single repository , making it easy to publish and collaborate. However, when it came to configuring the triggers of the Jenkins, it was easiest to have a single repository to listen to:

This meant that we made the choice to separate each node in the dependency tree into its own git repository - most of them under the SouthAfricaDigitalScience Github organisation . This made testing the components easier, as well as the creation of the dependency tree - and while all of the repos were in one organisation, it may even be manageable - but as soon as we start including different repos, the complexity may become an inhibiting factor.

So we have a dilemma : In order to keep testing atomic, we need to have individual repositories for nodes in the dependency graph, but in order to present a coherent code base to promote collaboratation, we need a single repository ?

Git submodules to the rescue.

Fortunately, this is an issue often encountered by software built on dependencies, with external plugins or similar. One of the most common solutions to the problem is to add the external bits as git submodules . Indeed from the git submodules documentation ² :

It often happens that while working on one project, you need to use another project from within it. Perhaps it’s a library that a third party developed or that you’re developing separately and using in multiple parent projects. A common issue arises in these scenarios: you want to be able to treat the two projects as separate yet still be able to use one from within the other.

This indeed seems like our case. We want to have a single point of entry for collaborators and applications - the CODE-RADE main repository , while still keeping the modularity which allows atomic testing and integration.

Onward, then !

So, I’ve started adding submodules to a subdirectory of the CODE-RADE repo, and will soon have all of them included. The CODE-RADE repo itself will not get tested, but it will be possible to fork it including the submodules and send pull requests. These will get tested only by the specific job which is configured to listen to changes in the submodule repository, i.e. if changes are made in the gcc-deploy submodule, and this pull request is sent to the CODE-RADE repo, only the gcc-deploy job will get triggered.

Modules, yo.

For more information, see the build server

References and Footnotes

 Tagged with blog  •  CODE-RADE  •  Release