Pipeline Host
To begin creating your pipeline, use the Pipeline.CreateBuilder() method.
Defining Modules
Defining Modules
Execution and Dependencies
The default behaviour is for modules to run in parallel, to speed up a pipeline as much as possible.
Sharing data across modules
Modules have been designed with data and sharing at its core.
Run conditions
Reusable run conditions implement IRunCondition:
Retry Policies
When creating modules, you can set a retry policy per module using the Configure() method. The retry policy uses Polly, so if you've used Polly before you should be familiar with how to use it.
Skipping Modules
Using ModuleConfiguration
Logging
IModuleLogger
Testing
Testing with Mocked File System
Using F#
ModularPipelines modules can be authored and executed from an F# project. Inherit
Generate build server files
The ModularPipelines package can create the minimal YAML needed to run a
Always Run
Module Run Types
Analyzers
Built-in
Cancellation Tokens
When you override a Module's ExecuteAsync method, you are provided a CancellationToken by the framework.
Categories
Sometimes we want to run only certain parts of a pipeline, or we might want to split a pipeline up into different targets. For instance, a test run, and then later on a deploy run. Categories can help achieve that.
Console Progress
If you are using an interactive terminal, then a progress dialog will be displayed to you. This will attempt to show you estimated remaining time, and the current progress of all executing modules.
Custom Commands
Many common CLI tools, such as npm, yarn, dotnet, docker, kubectl, have all had strong objects created to wrap around their CLI commands.
Generate a private CLI integration
ModularPipelines.OptionsGenerator is a .NET tool that generates the same strongly typed
Hooks
Module Hooks
Ignoring Failures
Sometimes a module might throw an exception, but we simply don't care as it's not that important, or a specific error might be expected.
Inheriting
Each 'Module' is expected to be registered only once. If you build a custom module that you'd like to instantiate multiple times but with different options, then you should create a new Module type that inherits from an abstract base module.
Parallelization
As mentioned, modules will try to run in parallel by default, waiting on any dependencies if they need to.
Pipeline Execution Modes
A pipeline has two execution modes:
Requirements
If you'd like to fail fast, you can register some Requirement classes that do some checks on start up to make sure things are as expected.
Secrets
Use IOptions\
Module History
What is it?
Sub-Modules
What are they?
Time Estimator
The time estimator is a class built by you, used to estimate times for modules for displaying in the console progress dialog. It isn't mandatory, but without it, estimated times will not be correct.
Timeouts
Modules have a 30-minute timeout by default. Configure the pipeline default when your workloads need a different limit, or use TimeSpan.Zero to disable it: