Where to Contribute
Screwdriver has a modular architecture, and the various responsibilities are split up into separate repos.
Check out the architecture diagram to see the overall workflow of Continuous Delivery using Screwdriver. The next few sections will help lay out where different code repositories fit.
Screwdriver API
The screwdriver repo is the core of screwdriver, providing the API endpoints for everything that screwdriver does. The API is based on the hapijs framework and is implemented in node as a series of plugins.
- The entry point is server.js
- Build bookends allows a user to customize setup and teardown steps for builds.
- The API can also uploading code coverage reports and/or test results. coverage-bookend defines the relationship between Screwdriver and coverage bookends. coverage-base is the base class for defining the behavior between Screwdriver coverage bookend plugins, like coverage-sonar.
Models
The object models provide the definition of the data that is saved in datastores (analogous to databases). This is done in two parts:
- data-schema: Schema definition with Joi
- models: Specific business logic around the data schema
Datastores
A datastore implementation is used as the interface between the API and a data storage mechanism. There are several implementations written in node around a common interface.
- datastore-base: Base class defining the interface for datastore implementations
- datastore-sequelize: MySQL, PostgreSQL, SQLite3, and MS SQL implementations
Source Code Management
An SCM implementation is used as the interface between the API and an SCM. There are several implementations written in nodejs around a common interface.
- scm-base: Common interface
- scm-bitbucket: Bitbucket.org implementation
- scm-github: GitHub implementation
- scm-gitlab: GitLab implementation
- sd-repo: A Go-based tool for executing the Repo workflow for
getCheckoutCommandin scm-github
The scm-router is a generic scm plugin that routes builds to a specified scm.
Notifications
The API can also send notifications to users.
- notifications-base: The base class for defining the behavior between Screwdriver and notifications plugins
- notifications-email: Email implementation
- notifications-slack: Slack implementation
Parsers
Parsers help validate and parse various flows in Screwdriver.
- config-parser: A node module for validating and parsing user’s
screwdriver.yamlconfigurations - workflow-parser: A node module for parsing and converting pipeline configuration into a workflow graph
Templates and Commands
Templates are snippets of predefined code that people can use to replace a job definition in their screwdriver.yaml. A template contains a series of predefined steps along with a selected Docker image.
- templates: A repo for all build templates
- template-main: The CLI for validating and publishing job templates
- template-validator: A tool used by the API to validate a job template
Commands are snippets of predefined code that people can use to replace a step definition in their screwdriver.yaml. A
command contains a series of predefined commands.
- command-validator: A tool used by the API to validate a command
- cmd-install-node: Shared command to install node.js using nvm
- cmd-docker-trigger: Shared command to trigger Docker build for master and a specified tag
- junit-reports: Shared command to parse Junit reports
Launcher
The launcher performs step execution and housekeeping internal to build containers. This is written in Go and mounted into build containers as a binary.
- sd-cmd: A Go-based CLI for sharing binaries which provides a single interface for executing a versioned command (via remote binary, docker image, or habitat package) during a Screwdriver build
- sd-packages: Builds, packages, and publishes skopeo and zstd and other custom binaries for Launcher
- sd-step: A Shared Step allows people to use the same packages and commands in all build containers, regardless of build environment
- meta-cli: A Go-based CLI for reading/writing information from the metadata
Queue
The queue-service is a highly available REST based queue service for Screwdriver to enqueue builds and process them. It makes use of Resque to add a queueing mechanism.
- buildcluster-queue-worker: An amqp connection manager implementation that consumes jobs from Rabbitmq queue
Executors
An executor is used to manage build containers for any given job. Several implementations of executors have been created. All are designed to follow a common interface. Executor implementations are written in node:
- executor-base: Common interface
- executor-docker: Docker implementation
- executor-j5s: Jenkins implementation
- executor-k8s: Kubernetes implementation
- executor-nomad: Nomad implementation
The executor-queue is a generic executor plugin that routes builds through a Redis queue. The executor-router is a generic executor plugin that routes builds to a specified executor.
Artifacts
The Artifact Store (not to be confused with the datastores mentioned above) is used for saving log outputs, shared steps, templates, test coverage, and any artifacts that are generated during a build.
- artifact-bookend: Bookend for uploading artifacts to the store
- cache-bookend: Bookend for uploading and downloading build cache
- log service: A Go tool for reading logs from the Launcher and uploading them to the store
- store-cli: A Go-based CLI for communicating with the Screwdriver store
UI
The UI is an Ember-based user interface of Screwdriver.
Guide and Homepage
The Guide is documentation! Everything you ever hoped to know about the Screwdriver project. The Homepage is the basic one-pager that powers Screwdriver.cd. The Community is where open source community meeting notes and docs are kept.
Miscellaneous Tools
Bootstrap SD
These repos help you get started running Screwdriver quickly.
- aws-build-cluster: Quickstart tool to provision necessary EKS cluster and cloud infrastructure resources required by Screwdriver build cluster on AWS
- hyperctl-image: Creates a minimal Docker image containing hyperctl and k8s-vm scripts (used in screwdriver-chart)
- in-a-box: Python-based executable for bringing up an entire Screwdriver instance (UI, API, and log store) locally
- screwdriver-chart: Bootstraps the whole Screwdriver ecosystem and also nginx ingress controller
- sd-local: A Go-based tool for running a more feature-complete local Screwdriver
- sonar-pipeline: Pipeline to deploy SonarQube server to Kubernetes
Other
- circuit-fuses: Wrapper to provide a node-circuitbreaker with callback interface
- gitversion: Go-based tool for updating git tags on a repo for a new version number
- keymbinatorial: Generates the unique combinations of key values by taking a single value from each keys array
- logger: Common logging provider for Screwdriver components
- noop-container: Bare minimum Docker container for running builds
- raptor: Load test scripts for Screwdriver API
- sd-housekeeping: Repo containing House Keeping scripts like a bulk pipeline validator
- toolbox: Repository for handy Screwdriver-related scripts and other tools
Adding a New Screwdriver Repo
We have some tools to help start out new repos for Screwdriver:
- generator-screwdriver: Yeoman generator that bootstraps new repos for screwdriver
- eslint-config-screwdriver: Our ESLint rules for node-based code. Included in each new repo as part of the bootstrap process
If you create a new repo, please come back and edit this page so that others can know where your repo fits in.
Tests and Examples
The organization screwdriver-cd-test contains various example repos/screwdriver.yamls and acceptance tests for Screwdriver.cd.