What are Workflows

Workflows are a way to orchestrate and coordinate multiple jobs within a CircleCI pipeline. They provide a flexible and configurable structure to define the order and dependencies of jobs, allowing developers to create more advanced pipelines with parallel execution, conditional steps, and fan-in/fan-out patterns.

Key Components

There are three primary components of a CircleCI Workflow:

• Jobs: Independent tasks that perform a specific operation, such as building, testing, or deploying code.
• Workflows: A set of rules that define the order and dependencies of jobs within a pipeline.
• Configuration: A YAML file (.circleci/config.yml) that contains the pipeline configuration, including job and workflow definitions.

Creating a Simple Workflow

To start with, let's create a simple workflow that consists of three jobs: build, test, and deploy.

Configuring your CircleCI Project

1. Sign up for a CircleCI account or log in if you already have one.
2. Connect your version control system (GitHub or Bitbucket) to CircleCI.
3. Add your project to CircleCI by selecting it from the list of repositories.

Defining a Basic Workflow

In your project's repository, create a .circleci directory and a config.yml file inside it. This file will contain the configuration for your pipeline, including job and workflow definitions.

version: 2.1
jobs:
  build:
    # Job configuration for the "build" job
  test:
    # Job configuration for the "test" job
  deploy:
    # Job configuration for the "deploy" job

workflows:
  version: 2
  my_workflow:
    jobs:
      - build
      - test:
          requires:
            - build
      - deploy:
          requires:
          - test

Running and Monitoring your Workflow

1. Push your changes to your repository.
2. CircleCI will automatically detect the new configuration and start running your workflow.

To monitor the progress of your workflow, navigate to the CircleCI Dashboard and select your project. You will see an overview of your pipeline's execution, including the status of each job.

Parallelism and Fan-in/Fan-out

Workflows in CircleCI allow you to take advantage of parallelism and fan-in/fan-out patterns to optimize the execution time of your CI/CD pipeline.

Expediting Your Builds with Parallelism

Parallelism in CircleCI involves running multiple instances of a job simultaneously to process tasks faster. You can configure the level of parallelism for each job in your workflow using the parallelism key.

For example, if you have a job that runs tests and you want to split the test suite into four parallel tasks, you can do so by setting the parallelism key to 4:

jobs:
  test:
    parallelism: 4
    # Job configuration for the "test" job

Understanding Fan-in and Fan-out

Fan-out is a pattern where a single job triggers multiple downstream jobs. In contrast, fan-in is when multiple jobs converge into a single downstream job. These patterns allow you to optimize your CI/CD pipeline by running jobs in parallel when possible and consolidating the results when needed.

Implementing Parallelism, Fan-in, and Fan-out in Workflows

To implement parallelism, fan-in, and fan-out in your CircleCI workflow, you can use the requires key to define job dependencies. By specifying the jobs that must complete before a given job can run, you can create complex workflows that optimize the execution of your pipeline.

For example, consider a workflow with the following jobs:

• build
• test_a, test_b, and test_c (running in parallel)
• ``deploy`

You can define this workflow as follows:

workflows:
  version: 2
  my_workflow:
    jobs:
      - build
      - test_a:
          requires:
            - build
      - test_b:
          requires:
            - build
      - test_c:
          requires:
            - build
      - deploy:
          requires:
            - test_a
            - test_b
            - test_c

Conditional Workflows and Approvals

CircleCI workflows support conditional execution and manual approvals, allowing you to have more control over your pipeline.

Defining Conditional Workflows

You can create conditional workflows using the when and unless keys along with a set of predefined conditions or custom logic. For example, you can run a specific job only when a particular branch is pushed:

workflows:
  version: 2
  my_workflow:
    jobs:
      - build
      - deploy:
          requires:
            - build
          filters:
            branches:
              only: /^release-.*/

Implementing Approval Steps

Manual approval steps in CircleCI allow you to pause a workflow and wait for user approval before continuing. To add an approval step to your workflow, use the type: approval key:

workflows:
  version: 2
  my_workflow:
    jobs:
      - build
      - test
      - hold_for_approval:
          type: approval
          requires:
            - test
      - deploy:
          requires:
            - hold_for_approval

Advanced Conditional Workflows

You can also create more complex conditional workflows using CircleCI's pipeline parameters and custom logic. Pipeline parameters are key-value pairs that you can pass into your configuration file and use them to define conditional logic for your workflows. To create a pipeline parameter, add the parameters key under the version key in your config.yml file:

version: 2.1
parameters:
  deploy_production:
    type: boolean
    default: false

jobs:
  # Job definitions

workflows:
  version: 2
  my_workflow:
    jobs:
      - build
      - test
      - deploy_staging
      - hold_for_approval:
          type: approval
          requires:
            - deploy_staging
      - deploy_production:
          requires:
            - hold_for_approval
          when: << pipeline.parameters.deploy_production >>

In this example, the deploy_production job will only run if the deploy_production pipeline parameter is set to true. You can pass this parameter when triggering the pipeline via the CircleCI API.

References

https://circleci.com/docs/workflows/