index.md 14.2 KB
Newer Older
1 2
# Serverless

Evan Read's avatar
Evan Read committed
3 4
> Introduced in GitLab 11.5.

danielgruesso's avatar
danielgruesso committed
5 6
CAUTION: **Caution:**
Serverless is currently in [alpha](https://about.gitlab.com/handbook/product/#alpha).
Daniel Gruesso's avatar
Daniel Gruesso committed
7

8 9 10 11
Run serverless workloads on Kubernetes using [Knative](https://cloud.google.com/knative/).

## Overview

Daniel Gruesso's avatar
Daniel Gruesso committed
12
Knative extends Kubernetes to provide a set of middleware components that are useful to build modern, source-centric, container-based applications. Knative brings some significant benefits out of the box through its main components:
13

Daniel Gruesso's avatar
Daniel Gruesso committed
14 15 16
- [Build](https://github.com/knative/build): Source-to-container build orchestration.
- [Eventing](https://github.com/knative/eventing): Management and delivery of events.
- [Serving](https://github.com/knative/serving): Request-driven compute that can scale to zero.
17 18 19

For more information on Knative, visit the [Knative docs repo](https://github.com/knative/docs).

Daniel Gruesso's avatar
Daniel Gruesso committed
20 21
With GitLab serverless, you can deploy both functions-as-a-service (FaaS) and serverless applications.

22
## Prerequisites
23 24 25

To run Knative on Gitlab, you will need:

26 27 28 29 30
1. **Existing GitLab project:** You will need a GitLab project to associate all resources. The simplest way to get started:

    - If you are planning on deploying functions, clone the [functions example project](https://gitlab.com/knative-examples/functions) to get started.
    - If you are planning on deploying a serverless application, clone the sample [Knative Ruby App](https://gitlab.com/knative-examples/knative-ruby-app) to get started.

Daniel Gruesso's avatar
Daniel Gruesso committed
31 32
1. **Kubernetes Cluster:** An RBAC-enabled Kubernetes cluster is required to deploy Knative.
    The simplest way to get started is to add a cluster using [GitLab's GKE integration](../index.md#adding-and-creating-a-new-gke-cluster-via-gitlab).
Mike Lewis's avatar
Mike Lewis committed
33
    The set of minimum recommended cluster specifications to run Knative is 3 nodes, 6 vCPUs, and 22.50 GB memory.
Daniel Gruesso's avatar
Daniel Gruesso committed
34 35
1. **Helm Tiller:** Helm is a package manager for Kubernetes and is required to install
    Knative.
Daniel Gruesso's avatar
Daniel Gruesso committed
36 37
1. **GitLab Runner:** A runner is required to run the CI jobs that will deploy serverless
    applications or functions onto your cluster. You can install the GitLab Runner
Mike Lewis's avatar
Mike Lewis committed
38
    onto the existing Kubernetes cluster. See [Installing Applications](../index.md#installing-applications) for more information.
Daniel Gruesso's avatar
Daniel Gruesso committed
39 40 41
1. **Domain Name:** Knative will provide its own load balancer using Istio. It will provide an
    external IP address for all the applications served by Knative. You will be prompted to enter a
    wildcard domain where your applications will be served. Configure your DNS server to use the
42
    external IP address for that domain.
43
1. **`.gitlab-ci.yml`:** GitLab uses [Kaniko](https://github.com/GoogleContainerTools/kaniko)
Daniel Gruesso's avatar
Daniel Gruesso committed
44
    to build the application and the [TriggerMesh CLI](https://github.com/triggermesh/tm) to simplify the
danielgruesso's avatar
danielgruesso committed
45
    deployment of knative services and functions.
46
1. **`serverless.yml`** (for [functions only](#deploying-functions)): When using serverless to deploy functions, the `serverless.yml` file
Daniel Gruesso's avatar
Daniel Gruesso committed
47 48
    will contain the information for all the functions being hosted in the repository as well as a reference to the
    runtime being used.
49
1. **`Dockerfile`** (for [applications only](#deploying-serverless-applications): Knative requires a `Dockerfile` in order to build your application. It should be included
Daniel Gruesso's avatar
Daniel Gruesso committed
50
    at the root of your project's repo and expose port `8080`.
51 52
1. **Prometheus** (optional): Installing Prometheus allows you to monitor the scale and traffic of your serverless function/application.
    See [Installing Applications](../index.md#installing-applications) for more information.
53 54 55 56

## Installing Knative via GitLab's Kubernetes integration

NOTE: **Note:**
Daniel Gruesso's avatar
Daniel Gruesso committed
57
The minimum recommended cluster size to run Knative is 3-nodes, 6 vCPUs, and 22.50 GB memory. **RBAC must be enabled.**
58

Daniel Gruesso's avatar
Daniel Gruesso committed
59 60
1. [Add a Kubernetes cluster](../index.md) and [install Helm](../index.md#installing-applications).
1. Once Helm has been successfully installed, scroll down to the Knative app section. Enter the domain to be used with
Mike Lewis's avatar
Mike Lewis committed
61
    your application/functions (e.g. `example.com`) and click **Install**.
62 63 64

    ![install-knative](img/install-knative.png)

Daniel Gruesso's avatar
Daniel Gruesso committed
65
1. After the Knative installation has finished, you can wait for the IP address to be displayed in the
66
   **Knative IP Address** field (takes up to 5 minutes) or retrieve the Istio Ingress IP address by running the following command:
67

Daniel Gruesso's avatar
Daniel Gruesso committed
68 69 70
   ```bash
   kubectl get svc --namespace=istio-system knative-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip} '
   ```
71

Daniel Gruesso's avatar
Daniel Gruesso committed
72
   Output:
73

Daniel Gruesso's avatar
Daniel Gruesso committed
74 75 76
   ```bash
   35.161.143.124 my-machine-name:~ my-user$
   ```
77

78 79 80 81 82
   NOTE: **Note:**
   Running `kubectl` commands on your cluster requires setting up access to the cluster first.
   For clusters created on GKE, see [GKE Cluster Access](https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl),
   for other platforms [Install kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/).

Daniel Gruesso's avatar
Daniel Gruesso committed
83 84
1. The ingress is now available at this address and will route incoming requests to the proper service based on the DNS
   name in the request. To support this, a wildcard DNS A record should be created for the desired domain name. For example,
Daniel Gruesso's avatar
Daniel Gruesso committed
85
   if your Knative base domain is `example.com` then you need to create an A record with domain `*.example.com`
Daniel Gruesso's avatar
Daniel Gruesso committed
86
   pointing the ip address of the ingress.
87

danielgruesso's avatar
danielgruesso committed
88
    ![dns entry](img/dns-entry.png)
89

Daniel Gruesso's avatar
Daniel Gruesso committed
90 91 92 93
NOTE: **Note:**
You can deploy either [functions](#deploying-functions) or [serverless applications](#deploying-serverless-applications)
on a given project but not both. The current implementation makes use of a `serverless.yml` file to signal a FaaS project.

Mike Lewis's avatar
Mike Lewis committed
94
## Deploying functions
Daniel Gruesso's avatar
Daniel Gruesso committed
95 96 97

> Introduced in GitLab 11.6.

danielgruesso's avatar
danielgruesso committed
98
Using functions is useful for dealing with independent
Daniel Gruesso's avatar
Daniel Gruesso committed
99 100 101
events without needing to maintain a complex unified infrastructure. This allows
you to focus on a single task that can be executed/scaled automatically and independently.

danielgruesso's avatar
danielgruesso committed
102
Currently the following [runtimes](https://gitlab.com/triggermesh/runtimes) are offered:
103 104 105 106

- node.js
- kaniko

Daniel Gruesso's avatar
Daniel Gruesso committed
107
You can find and import all the files referenced in this doc in the **[functions example project](https://gitlab.com/knative-examples/functions)**.
108

109
Follow these steps to deploy a function using the Node.js runtime to your Knative instance (you can skip these steps if you've cloned the example project):
110 111 112

1. Create a directory that will house the function. In this example we will create a directory called `echo` at the root of the project.

danielgruesso's avatar
danielgruesso committed
113 114 115
1. Create the file that will contain the function code. In this example, our file is called `echo.js` and is located inside the `echo` directory. If your project is:
    - Public, continue to the next step.
    - Private, you will need to [create a GitLab deploy token](../../deploy_tokens/index.md#creating-a-deploy-token) with `gitlab-deploy-token` as the name and the `read_registry` scope.
Daniel Gruesso's avatar
Daniel Gruesso committed
116

117 118
1. `.gitlab-ci.yml`: this defines a pipeline used to deploy your functions.
   It must be included at the root of your repository:
Daniel Gruesso's avatar
Daniel Gruesso committed
119 120

   ```yaml
121 122
   include:
     template: Serverless.gitlab-ci.yml
Daniel Gruesso's avatar
Daniel Gruesso committed
123 124

   functions:
125 126
     extends: .serverless:deploy:functions
     environment: production
Daniel Gruesso's avatar
Daniel Gruesso committed
127 128
   ```

129 130 131 132 133 134 135 136
    This `.gitlab-ci.yml` creates a `functions` job that invokes some
    predefined commands to deploy your functions to Knative.

    `Serverless.gitlab-ci.yml` is a template that allows customization.
    You can either import it with `include` parameter and use `extends` to
    customize your jobs, or you can inline entire template by choosing it
    from "Apply a template" dropdown when editing `.gitlab-ci.yml` file through
    the User Interface.
137 138 139

2. `serverless.yml`: this file contains the metadata for your functions,
   such as name, runtime, and environment.
140

141 142 143 144 145
   It must be included at the root of your repository.
   The following is a sample `echo` function which shows the required structure
   for the file.

   You can find the relevant files for this project in the [functions example project](https://gitlab.com/knative-examples/functions).
146

Daniel Gruesso's avatar
Daniel Gruesso committed
147
   ```yaml
148
   service: my-functions
Daniel Gruesso's avatar
Daniel Gruesso committed
149 150 151 152 153 154 155 156 157
   description: "Deploying functions from GitLab using Knative"

   provider:
     name: triggermesh
     registry-secret: gitlab-registry
     environment:
       FOO: BAR

   functions:
158
     echo:
159 160 161
       handler: echo
       runtime: https://gitlab.com/triggermesh/runtimes/raw/master/nodejs.yaml
       description: "echo function using node.js runtime"
Daniel Gruesso's avatar
Daniel Gruesso committed
162
       buildargs:
163 164 165
        - DIRECTORY=echo
       environment:
        FUNCTION: echo
Daniel Gruesso's avatar
Daniel Gruesso committed
166 167
   ```

Evan Read's avatar
Evan Read committed
168 169
The `serverless.yml` file references both an `echo` directory (under `buildargs`) and an `echo` file (under `handler`),
which is a reference to `echo.js` in the [repository](https://gitlab.com/knative-examples/functions). Additionally, it
170
contains three sections with distinct parameters:
171

danielgruesso's avatar
danielgruesso committed
172
### `service`
173

danielgruesso's avatar
danielgruesso committed
174 175
| Parameter | Description |
|-----------|-------------|
176 177
| `service` | Name for the Knative service which will serve the function. |
| `description` | A short description of the `service`. |
178

danielgruesso's avatar
danielgruesso committed
179
### `provider`
180

danielgruesso's avatar
danielgruesso committed
181 182
| Parameter | Description |
|-----------|-------------|
183
| `name` | Indicates which provider is used to execute the `serverless.yml` file. In this case, the TriggerMesh `tm` CLI. |
danielgruesso's avatar
danielgruesso committed
184
| `registry-secret` | Indicates which registry will be used to store docker images. The sample function is using the GitLab Registry (`gitlab-registry`). A different registry host may be specified using `registry` key in the `provider` object. If changing the default, update the permission and the secret value on the `gitlab-ci.yml` file |
185
| `environment` | Includes the environment variables to be passed as part of function execution for **all** functions in the file, where `FOO` is the variable name and `BAR` are he variable contents. You may replace this with you own variables. |
186

danielgruesso's avatar
danielgruesso committed
187 188
### `functions`

189
In the `serverless.yml` example above, the function name is `echo` and the subsequent lines contain the function attributes.
190

danielgruesso's avatar
danielgruesso committed
191 192
| Parameter | Description |
|-----------|-------------|
193 194 195 196 197
| `handler` | The function's file name. In the example above, both the function name and the handler are the same. |
| `runtime` | The runtime to be used to execute the function. |
| `description` | A short description of the function. |
| `buildargs` | Pointer to the function file in the repo. In the sample the function is located in the `echo` directory. |
| `environment` | Sets an environment variable for the specific function only. |
198

Evan Read's avatar
Evan Read committed
199
After the `gitlab-ci.yml` template has been added and the `serverless.yml` file has been
200
created, pushing a commit to your project will result in a
Daniel Gruesso's avatar
Daniel Gruesso committed
201 202 203 204 205 206
CI pipeline being executed which will deploy each function as a Knative service.
Once the deploy stage has finished, additional details for the function will
appear under **Operations > Serverless**.

![serverless page](img/serverless-page.png)

207 208
This page contains all functions available for the project, the description for
accessing the function, and, if available, the function's runtime information.
Daniel Gruesso's avatar
Daniel Gruesso committed
209
The details are derived from the Knative installation inside each of the project's
210
Kubernetes cluster. Click on each function to obtain detailed scale and invocation data.
Daniel Gruesso's avatar
Daniel Gruesso committed
211 212 213 214 215 216 217

The function details can be retrieved directly from Knative on the cluster:

```bash
kubectl -n "$KUBE_NAMESPACE" get services.serving.knative.dev
```

218
The sample function can now be triggered from any HTTP client using a simple `POST` call:
219

220
  1. Using curl (replace the URL on the last line with the URL of your application):
221 222 223 224 225 226

      ```bash
      curl \
      --header "Content-Type: application/json" \
      --request POST \
      --data '{"GitLab":"FaaS"}' \
227
      <http://functions-echo.functions-1.functions.example.com/>
228 229 230
      ```
  2. Using a web-based tool (ie. postman, restlet, etc)

Takuya Noguchi's avatar
Takuya Noguchi committed
231
      ![function execution](img/function-execution.png)
232

Daniel Gruesso's avatar
Daniel Gruesso committed
233 234 235 236 237
## Deploying Serverless applications

> Introduced in GitLab 11.5.

NOTE: **Note:**
Daniel Gruesso's avatar
Daniel Gruesso committed
238
You can reference and import the sample [Knative Ruby App](https://gitlab.com/knative-examples/knative-ruby-app) to get started.
Daniel Gruesso's avatar
Daniel Gruesso committed
239 240

Add the following `.gitlab-ci.yml` to the root of your repository
241
(you may skip this step if you've previously cloned the sample [Knative Ruby App](https://gitlab.com/knative-examples/knative-ruby-app) mentioned above):
Daniel Gruesso's avatar
Daniel Gruesso committed
242 243

```yaml
244 245
include:
  template: Serverless.gitlab-ci.yml
Daniel Gruesso's avatar
Daniel Gruesso committed
246 247

build:
248
  extends: .serverless:build:image
Daniel Gruesso's avatar
Daniel Gruesso committed
249 250

deploy:
251
  extends: .serverless:deploy:image
Daniel Gruesso's avatar
Daniel Gruesso committed
252 253
```

254
`Serverless.gitlab-ci.yml` is a template that allows customization.
255
You can either import it with `include` parameter and use `extends` to
256
customize your jobs, or you can inline the entire template by choosing it
257
from **Apply a template** dropdown when editing the `.gitlab-ci.yml` file through
258
the user interface.
259

260
### Deploy the application with Knative
261

262 263
With all the pieces in place, the next time a CI pipeline runs, the Knative application will be deployed. Navigate to
**CI/CD > Pipelines** and click the most recent pipeline.
264

265
### Obtain the URL for the Knative deployment
266

267
Go to the **CI/CD > Pipelines** and click on the pipeline that deployed your app. Once all the stages of the pipeline finish, click the **deploy** stage.
268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288

![deploy stage](img/deploy-stage.png)

The output will look like this:

```bash
Running with gitlab-runner 11.5.0~beta.844.g96d88322 (96d88322)
  on docker-auto-scale 72989761
Using Docker executor with image gcr.io/triggermesh/tm@sha256:e3ee74db94d215bd297738d93577481f3e4db38013326c90d57f873df7ab41d5 ...
Pulling docker image gcr.io/triggermesh/tm@sha256:e3ee74db94d215bd297738d93577481f3e4db38013326c90d57f873df7ab41d5 ...
Using docker image sha256:6b3f6590a9b30bd7aafb9573f047d930c70066e43955b4beb18a1eee175f6de1 for gcr.io/triggermesh/tm@sha256:e3ee74db94d215bd297738d93577481f3e4db38013326c90d57f873df7ab41d5 ...
Running on runner-72989761-project-4342902-concurrent-0 via runner-72989761-stg-srm-1541795796-27929c96...
Cloning repository...
Cloning into '/builds/danielgruesso/knative'...
Checking out 8671ad20 as master...
Skipping Git submodules setup
$ echo "$CI_REGISTRY_IMAGE"
registry.staging.gitlab.com/danielgruesso/knative
$ tm -n "$KUBE_NAMESPACE" --config "$KUBECONFIG" deploy service "$CI_PROJECT_NAME" --from-image "$CI_REGISTRY_IMAGE" --wait
Deployment started. Run "tm -n knative-4342902 describe service knative" to see the details
Waiting for ready state.......
289
Service domain: knative.knative-4342902.example.com
290 291 292
Job succeeded
```

Daniel Gruesso's avatar
Daniel Gruesso committed
293
The second to last line, labeled **Service domain** contains the URL for the deployment. Copy and paste the domain into your
294 295
browser to see the app live.

Daniel Gruesso's avatar
Daniel Gruesso committed
296
![knative app](img/knative-app.png)