Skip to content

GitLab

Commit 8760e120 authored by Marcia Ramos's avatar Marcia Ramos
Browse files

Merge branch 'docs/autodevops-multiple-clusters' into 'master' 

Port the Auto DevOps docs from EE

See merge request gitlab-org/gitlab-ce!19539
parents 3094f1c1 135dc2c5
Hide whitespace changes
Inline Side-by-side
Showing with 100 additions and 33 deletions
doc/topics/autodevops/index.md
View file @ 8760e120
......@@ -41,6 +41,7 @@ project in an easy and automatic way:
1. [Auto Code Quality](#auto-code-quality)
1. [Auto SAST (Static Application Security Testing)](#auto-sast)
1. [Auto Dependency Scanning](#auto-dependency-scanning)
1. [Auto License Management](#auto-license-management)
1. [Auto Container Scanning](#auto-container-scanning)
1. [Auto Review Apps](#auto-review-apps)
1. [Auto DAST (Dynamic Application Security Testing)](#auto-dast)
......@@ -62,7 +63,7 @@ Auto DevOps provides great defaults for all the stages; you can, however,
For an overview on the creation of Auto DevOps, read the blog post [From 2/3 of the Self-Hosted Git Market, to the Next-Generation CI System, to Auto DevOps](https://about.gitlab.com/2017/06/29/whats-next-for-gitlab-ci/).
## Prerequisites
## Requirements
TIP: **Tip:**
For self-hosted installations, the easiest way to make use of Auto DevOps is to
......@@ -112,25 +113,26 @@ NOTE: **Note:**
If you do not have Kubernetes or Prometheus installed, then Auto Review Apps,
Auto Deploy, and Auto Monitoring will be silently skipped.
### Auto DevOps base domain
## Auto DevOps base domain
The Auto DevOps base domain is required if you want to make use of [Auto
Review Apps](#auto-review-apps) and [Auto Deploy](#auto-deploy). It is defined
either under the project's CI/CD settings while
[enabling Auto DevOps](#enabling-auto-devops) or in instance-wide settings in
the CI/CD section.
It can also be set at the project or group level as a variable, `AUTO_DEVOPS_DOMAIN`.
Review Apps](#auto-review-apps) and [Auto Deploy](#auto-deploy). It can be defined
in three places:
A wildcard DNS A record matching the base domain is required, for example,
- either under the project's CI/CD settings while [enabling Auto DevOps](#enabling-auto-devops)
- or in instance-wide settings in the **admin area > Settings** under the "Continuous Integration and Delivery" section
- or at the project or group level as a variable: `AUTO_DEVOPS_DOMAIN` (required if you want to use [multiple clusters](#using-multiple-kubernetes-clusters))
A wildcard DNS A record matching the base domain(s) is required, for example,
given a base domain of `example.com`, you'd need a DNS entry like:
```
*.example.com 3600 A 1.2.3.4
```
where `example.com` is the domain name under which the deployed apps will be served,
In this case, `example.com` is the domain name under which the deployed apps will be served,
and `1.2.3.4` is the IP address of your load balancer; generally NGINX
([see prerequisites](#prerequisites)). How to set up the DNS record is beyond
([see requirements](#requirements)). How to set up the DNS record is beyond
the scope of this document; you should check with your DNS provider.
Alternatively you can use free public services like [xip.io](http://xip.io) or
......@@ -146,6 +148,56 @@ If GitLab is installed using the [GitLab Omnibus Helm Chart], there are two
options: provide a static IP, or have one assigned. For more information see the
relevant docs on the [network prerequisites](../../install/kubernetes/gitlab_omnibus.md#networking-prerequisites).
## Using multiple Kubernetes clusters **[PREMIUM]**
When using Auto DevOps, you may want to deploy different environments to
different Kubernetes clusters. This is possible due to the 1:1 connection that
[exists between them](../../user/project/clusters/index.md#multiple-kubernetes-clusters).
In the [Auto DevOps template](https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Auto-DevOps.gitlab-ci.yml)
(used behind the scenes by Auto DevOps), there are currently 3 defined environment names that you need to know:
- `review/` (every environment starting with `review/`)
- `staging`
- `production`
Those environments are tied to jobs that use [Auto Deploy](#auto-deploy), so
except for the environment scope, they would also need to have a different
domain they would be deployed to. This is why you need to define a separate
`AUTO_DEVOPS_DOMAIN` variable for all the above
[based on the environment](../../ci/variables/README.md#limiting-environment-scopes-of-variables).
The following table is an example of how the three different clusters would
be configured.
| Cluster name | Cluster environment scope | `AUTO_DEVOPS_DOMAIN` variable value | Variable environment scope | Notes |
| ------------ | -------------- | ----------------------------- | ------------- | ------ |
| review | `review/*` | `review.example.com` | `review/*` | The review cluster which will run all [Review Apps](../../ci/review_apps/index.md). `*` is a wildcard, which means it will be used by every environment name starting with `review/`. |
| staging | `staging` | `staging.example.com` | `staging` | (Optional) The staging cluster which will run the deployments of the staging environments. You need to [enable it first](#deploy-policy-for-staging-and-production-environments). |
| production | `production` | `example.com` | `production` | The production cluster which will run the deployments of the production environment. You can use [incremental rollouts](#incremental-rollout-to-production). |
To add a different cluster for each environment:
1. Navigate to your project's **Operations > Kubernetes** and create the Kubernetes clusters
with their respective environment scope as described from the table above.
![Auto DevOps multiple clusters](img/autodevops_multiple_clusters.png)
1. After the clusters are created, navigate to each one and install Helm Tiller
and Ingress.
1. Make sure you have [configured your DNS](#auto-devops-base-domain) with the
specified Auto DevOps domains.
1. Navigate to your project's **Settings > CI/CD > Variables** and add
the `AUTO_DEVOPS_DOMAIN` variables with their respective environment
scope.
![Auto DevOps domain variables](img/autodevops_domain_variables.png)
Now that all is configured, you can test your setup by creating a merge request
and verifying that your app is deployed as a review app in the Kubernetes
cluster with the `review/*` environment scope. Similarly, you can check the
other environments.
## Quick start
If you are using GitLab.com, see our [quick start guide](quick_start_guide.md)
......@@ -154,13 +206,13 @@ Google Cloud.
## Enabling Auto DevOps
If you haven't done already, read the [prerequisites](#prerequisites) to make
If you haven't done already, read the [requirements](#requirements) to make
full use of Auto DevOps. If this is your fist time, we recommend you follow the
[quick start guide](#quick-start).
[quick start guide](quick_start_guide.md).
To enable Auto DevOps to your project:
1. Check that your project doesn't have a `.gitlab-ci.yml`, and remove it otherwise
1. Check that your project doesn't have a `.gitlab-ci.yml`, or remove it otherwise
1. Go to your project's **Settings > CI/CD > General pipelines settings** and
find the Auto DevOps section
1. Select "Enable Auto DevOps"
......@@ -230,7 +282,7 @@ In GitLab Starter, differences between the source and
target branches are also
[shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality_diff.html).
### Auto SAST
### Auto SAST **[ULTIMATE]**
> Introduced in [GitLab Ultimate][ee] 10.3.
......@@ -241,9 +293,9 @@ report is created, it's uploaded as an artifact which you can later download and
check out.
In GitLab Ultimate, any security warnings are also
[shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/sast.html).
[shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/sast.html).
### Auto Dependency Scanning
### Auto Dependency Scanning **[ULTIMATE]**
> Introduced in [GitLab Ultimate][ee] 10.7.
......@@ -254,7 +306,20 @@ report is created, it's uploaded as an artifact which you can later download and
check out.
In GitLab Ultimate, any security warnings are also
[shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/dependency_scanning.html).
[shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/dependency_scanning.html).
### Auto License Management **[ULTIMATE]**
> Introduced in [GitLab Ultimate][ee] 11.0.
License Management uses the
[License Management Docker image](https://gitlab.com/gitlab-org/security-products/license_management)
to search the project dependencies for their license. Once the
report is created, it's uploaded as an artifact which you can later download and
check out.
In GitLab Ultimate, any licenses are also
[shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/license_management.html).
### Auto Container Scanning
......@@ -267,13 +332,13 @@ created, it's uploaded as an artifact which you can later download and
check out.
In GitLab Ultimate, any security warnings are also
[shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/container_scanning.html).
[shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/container_scanning.html).
### Auto Review Apps
NOTE: **Note:**
This is an optional step, since many projects do not have a Kubernetes cluster
available. If the [prerequisites](#prerequisites) are not met, the job will
available. If the [requirements](#requirements) are not met, the job will
silently be skipped.
CAUTION: **Caution:**
......@@ -295,7 +360,7 @@ up in the merge request widget for easy discovery. When the branch is deleted,
for example after the merge request is merged, the Review App will automatically
be deleted.
### Auto DAST
### Auto DAST **[ULTIMATE]**
> Introduced in [GitLab Ultimate][ee] 10.4.
......@@ -306,9 +371,9 @@ issues. Once the report is created, it's uploaded as an artifact which you can
later download and check out.
In GitLab Ultimate, any security warnings are also
[shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/dast.html).
[shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/dast.html).
### Auto Browser Performance Testing
### Auto Browser Performance Testing **[PREMIUM]**
> Introduced in [GitLab Premium][ee] 10.4.
......@@ -320,13 +385,14 @@ Auto Browser Performance Testing utilizes the [Sitespeed.io container](https://h
/direction
```
In GitLab Premium, performance differences between the source and target branches are [shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/browser_performance_testing.html).
In GitLab Premium, performance differences between the source
and target branches are [shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/browser_performance_testing.html).
### Auto Deploy
NOTE: **Note:**
This is an optional step, since many projects do not have a Kubernetes cluster
available. If the [prerequisites](#prerequisites) are not met, the job will
available. If the [requirements](#requirements) are not met, the job will
silently be skipped.
CAUTION: **Caution:**
......@@ -363,7 +429,7 @@ executed somewhere else, it cannot be accessed again.
### Auto Monitoring
NOTE: **Note:**
Check the [prerequisites](#prerequisites) for Auto Monitoring to make this stage
Check the [requirements](#requirements) for Auto Monitoring to make this stage
work.
Once your application is deployed, Auto Monitoring makes it possible to monitor
......@@ -437,7 +503,7 @@ repo or by specifying a project variable:
file in it, Auto DevOps will detect the chart and use it instead of the [default
one](https://gitlab.com/charts/charts.gitlab.io/tree/master/charts/auto-deploy-app).
This can be a great way to control exactly how your application is deployed.
- **Project variable** - Create a [variable](../../ci/variables/README.md#variables)
- **Project variable** - Create a [project variable](../../ci/variables/README.md#secret-variables)
`AUTO_DEVOPS_CHART` with the URL of a custom chart to use.
### Customizing `.gitlab-ci.yml`
......@@ -493,22 +559,23 @@ also be customized, and you can easily use a [custom buildpack](#custom-buildpac
| `POSTGRES_PASSWORD` | The PostgreSQL password; defaults to `testing-password`. Set it to use a custom password. |
| `POSTGRES_DB` | The PostgreSQL database name; defaults to the value of [`$CI_ENVIRONMENT_SLUG`](../../ci/variables/README.md#predefined-variables-environment-variables). Set it to use a custom database name. |
| `BUILDPACK_URL` | The buildpack's full URL. It can point to either Git repositories or a tarball URL. For Git repositories, it is possible to point to a specific `ref`, for example `https://github.com/heroku/heroku-buildpack-ruby.git#v142` |
| `SAST_CONFIDENCE_LEVEL` | The minimum confidence level of security issues you want to be reported; `1` for Low, `2` for Medium, `3` for High; defaults to `3`.|
| `DEP_SCAN_DISABLE_REMOTE_CHECKS` | Whether remote Dependency Scanning checks are disabled; defaults to `"false"`. Set to `"true"` to disable checks that send data to GitLab central servers. [Read more about remote checks](https://gitlab.com/gitlab-org/security-products/dependency-scanning#remote-checks).|
| `STAGING_ENABLED` | From GitLab 10.8, this variable can be used to define a [deploy policy for staging and production environments](#deploy-policy-for-staging-and-production-environments). |
| `CANARY_ENABLED` | From GitLab 11.0, this variable can be used to define a [deploy policy for canary environments](#deploy-policy-for-canary-environments). |
| `INCREMENTAL_ROLLOUT_ENABLED`| From GitLab 10.8, this variable can be used to enable an [incremental rollout](#incremental-rollout-to-production) of your application for the production environment. |
| `TEST_DISABLED` | From GitLab 11.0, this variable can be used to disable the `test` job. If the variable is present, the job will not be created. |
| `CODE_QUALITY_DISABLED` | From GitLab 11.0, this variable can be used to disable the `code_quality` job. If the variable is present, the job will not be created. |
| `LICENSE_MANAGEMENT_DISABLED` | From GitLab 11.0, this variable can be used to disable the `license_management` job. If the variable is present, the job will not be created. |
| `CODEQUALITY_DISABLED` | From GitLab 11.0, this variable can be used to disable the `codequality` job. If the variable is present, the job will not be created. |
| `SAST_DISABLED` | From GitLab 11.0, this variable can be used to disable the `sast` job. If the variable is present, the job will not be created. |
| `DEPENDENCY_SCANNING_DISABLED` | From GitLab 11.0, this variable can be used to disable the `dependency_scanning` job. If the variable is present, the job will not be created. |
| `CONTAINER_SCANNING_DISABLED` | From GitLab 11.0, this variable can be used to disable the `container_scanning` job. If the variable is present, the job will not be created. |
| `CONTAINER_SCANNING_DISABLED` | From GitLab 11.0, this variable can be used to disable the `sast:container` job. If the variable is present, the job will not be created. |
| `REVIEW_DISABLED` | From GitLab 11.0, this variable can be used to disable the `review` and the manual `review:stop` job. If the variable is present, these jobs will not be created. |
| `DAST_DISABLED` | From GitLab 11.0, this variable can be used to disable the `dast` job. If the variable is present, the job will not be created. |
| `PERFORMANCE_DISABLED` | From GitLab 11.0, this variable can be used to disable the `performance` job. If the variable is present, the job will not be created. |
TIP: **Tip:**
Set up the replica variables using a
[project variable](../../ci/variables/README.md#variables)
[project variable](../../ci/variables/README.md#secret-variables)
and scale your application by just redeploying it!
CAUTION: **Caution:**
......@@ -583,7 +650,7 @@ staging environment and deploy to production manually. For this scenario, the
`STAGING_ENABLED` environment variable was introduced.
If `STAGING_ENABLED` is defined in your project (e.g., set `STAGING_ENABLED` to
`1` as a variable), then the application will be automatically deployed
`1` as a secret variable), then the application will be automatically deployed
to a `staging` environment, and a `production_manual` job will be created for
you when you're ready to manually deploy to production.
......@@ -596,7 +663,7 @@ A [canary environment](https://docs.gitlab.com/ee/user/project/canary_deployment
before any changes are deployed to production.
If `CANARY_ENABLED` is defined in your project (e.g., set `CANARY_ENABLED` to
`1` as a variable) then two manual jobs will be created:
`1` as a secret variable) then two manual jobs will be created:
- `canary` which will deploy the application to the canary environment
- `production_manual` which is to be used by you when you're ready to manually
......@@ -612,7 +679,7 @@ This will allow you to first check how the app is behaving, and later manually
increasing the rollout up to 100%.
If `INCREMENTAL_ROLLOUT_ENABLED` is defined in your project (e.g., set
`INCREMENTAL_ROLLOUT_ENABLED` to `1` as a variable), then instead of the
`INCREMENTAL_ROLLOUT_ENABLED` to `1` as a secret variable), then instead of the
standard `production` job, 4 different
[manual jobs](../../ci/pipelines.md#manual-actions-from-the-pipeline-graph)
will be created:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment