Pipeline Host
To begin creating your pipeline, use the Pipeline.CreateBuilder() method.
Command Line
Pass the application's args to Pipeline.CreateBuilder(args) to enable the
Defining Modules
Defining Modules
Execution and Dependencies
The default behaviour is for modules to run in parallel, to speed up a pipeline as much as possible.
Sharing data across modules
Modules have been designed with data and sharing at its core.
Run conditions
Reusable run conditions implement IRunCondition:
Retry Policies
When creating modules, you can configure retries per module using the Configure() method.
Skipping Modules
Using ModuleConfiguration
Logging
IModuleLogger
Testing
Install ModularPipelines.Testing to execute one module without starting the full
Using F#
ModularPipelines modules can be authored and executed from an F# project. Inherit
Generate build server files
The ModularPipelines package can create the minimal YAML needed to run a
Source generator diagnostics
Modular Pipelines generates registration metadata at compile time. A generator
OpenTelemetry
Modular Pipelines emits traces and metrics through the standard .NET
Always Run
Module Run Types
Analyzers
Modular Pipelines includes Roslyn analyzers that catch pipeline authoring mistakes at compile time. Each rule uses the MP#### ID family, so it can be configured with standard .NET analyzer settings.
Cancellation Tokens
When you override a Module's ExecuteAsync method, you are provided a CancellationToken by the framework.
Categories
Sometimes we want to run only certain parts of a pipeline, or we might want to split a pipeline up into different targets. For instance, a test run, and then later on a deploy run. Categories can help achieve that.
Console Progress
If you are using an interactive terminal, then a progress dialog will be displayed to you. This will attempt to show you estimated remaining time, and the current progress of all executing modules.
Custom Commands
Many common CLI tools, such as npm, yarn, dotnet, docker, kubectl, have all had strong objects created to wrap around their CLI commands.
Generate a private CLI integration
ModularPipelines.OptionsGenerator is a .NET tool that generates the same strongly typed
Hooks
Module lifecycle behavior has three extension points:
Ignoring Failures
Sometimes a module might throw an exception, but we simply don't care as it's not that important, or a specific error might be expected.
Inheriting
Each 'Module' is expected to be registered only once. If you build a custom module that you'd like to instantiate multiple times but with different options, then you should create a new Module type that inherits from an abstract base module.
Cache Module Results
Fingerprint-based module caching skips work when its declared inputs have not changed. A cache entry contains the typed module result and files declared with ProducesArtifact, so a hit can restore outputs before dependent modules start.
Trimming and Native AOT
The core ModularPipelines package supports trimmed and Native AOT C# applications
Parallelization
As mentioned, modules will try to run in parallel by default, waiting on any dependencies if they need to.
Pipeline Execution Modes
A pipeline has two execution modes:
Requirements
If you'd like to fail fast, you can register some Requirement classes that do some checks on start up to make sure things are as expected.
Run reports and history
ModularPipelines can write a schema-versioned JSON report after every pipeline run. Reports contain
Secrets
Use IOptions\
Module History
What is it?
Sub-Modules
What are they?
Time Estimator
The time estimator is a class built by you, used to estimate times for modules for displaying in the console progress dialog. It isn't mandatory, but without it, estimated times will not be correct.
Timeouts
Modules have a 30-minute timeout by default. Configure the pipeline default when your workloads need a different limit, or use TimeSpan.Zero to disable it: