TestProject Agent on Kubernetes

Use cases
Parallel tests execution.
Setup many agents in seconds.
Test different browser versions.
Save time in the execution of your test suites. 
Diagram  
Requirements
Kubernetes cluster
kubectl
TestProject Agent => v0.66.4 
Deployment
This YAML snippet spins up 3 deployments:
1.TestProject Agent
2.Standalone selenium with pre-installed Chrome
3.Standalone selenium with pre-installed Firefox
The TestProject Agent will automatically detect the Chrome & Firefox containers.
By setting the TP_API_KEY environment variable we instruct the Agent to automatically register TestProject Agent. The API key can be created in the TestProject Web Application (as described here).
1.For enhanced security, it's recommended to use Kubernetes secrets for setting the TP_API_KEY.
 Create file '.secret' with TP_API_KEY.
 Then create the testproject-secrets as following:

1
kubectl create secret generic testproject-secrets --from-literal "TP_API_KEY=$(cat .secret)"
Copied!
2. Save the code below as k8s-testproject.yaml , then execute that YAML file in your local Kubernetes environment.
1
kubectl apply -f k8s-testproject.yaml
Copied!
Content of k8s-testproject.yaml as following:
1
# To execute this yml file use this command in your local kubernetes environment `kubectl apply -f k8s-testproject.yaml
2
#***************************************
3
#SELENIUM STANDALONE CHROME
4
#***************************************
5
apiVersion: v1
6
kind: Service
7
metadata:
8
  name: chrome
9
  labels:
10
    name: selenium-standalone-chrome
11
spec:
12
  selector:
13
    app: selenium-standalone-chrome
14
  ports:
15
    - name: "chrome"
16
      protocol: TCP    
17
      port: 4444
18
      targetPort: 4444
19
---
20
apiVersion: apps/v1
21
kind: Deployment
22
metadata:
23
  name: chrome
24
  labels:
25
    app: selenium-standalone-chrome
26
    name: selenium-standalone-chrome
27
spec:
28
  replicas: 1
29
  selector:
30
    matchLabels:
31
      app: selenium-standalone-chrome
32
  strategy:
33
    type: Recreate
34
  template:
35
    metadata:
36
      labels:
37
        app: selenium-standalone-chrome
38
        name: selenium-standalone-chrome
39
      labels:
40
        app: selenium-standalone-chrome
41
        name: selenium-standalone-chrome                              
42
    spec:
43
      restartPolicy: Always        
44
      containers:
45
        - image: selenium/standalone-chrome
46
          name: chrome       
47
          ports:
48
            - containerPort: 4444       
49
          volumeMounts:
50
            - name: dshm
51
              mountPath: /dev/shm
52
      volumes:
53
        - name: dshm
54
          emptyDir: { "medium": "Memory" }
55
---          
56
#***************************************
57
#SELENIUM STANDALONE FIREFOX
58
#*************************************** 
59
apiVersion: v1
60
kind: Service
61
metadata:
62
  name: firefox
63
  labels:
64
    name: selenium-standalone-firefox
65
spec:
66
  selector:
67
    app: selenium-standalone-firefox
68
  ports:
69
    - name: "firefox"
70
      protocol: TCP    
71
      port: 4444
72
      targetPort: 4444
73
---
74
apiVersion: apps/v1
75
kind: Deployment
76
metadata:
77
  name: firefox
78
  labels:
79
    app: selenium-standalone-firefox
80
    name: selenium-standalone-firefox
81
spec:
82
  replicas: 1
83
  selector:
84
    matchLabels:
85
      app: selenium-standalone-firefox
86
  strategy:
87
    type: Recreate
88
  template:
89
    metadata:
90
      labels:
91
        app: selenium-standalone-firefox
92
        name: selenium-standalone-firefox
93
      labels:
94
        app: selenium-standalone-firefox
95
        name: selenium-standalone-firefox                              
96
    spec:
97
      restartPolicy: Always        
98
      containers:
99
        - image: selenium/standalone-firefox
100
          name: firefox       
101
          ports:
102
            - containerPort: 4444
103
          volumeMounts:
104
            - name: dshm
105
              mountPath: /dev/shm
106
      volumes:
107
        - name: dshm
108
          emptyDir: { "medium": "Memory" }
109
---            
110
#***************************************
111
#TESTPROJECT AGENT
112
#***************************************           
113
apiVersion: v1
114
kind: Service
115
metadata:
116
  name: testproject-agent
117
  labels:
118
    name: testproject-agent
119
spec:
120
  selector:
121
    app: testproject-agent
122
  ports:
123
    - name: "8585"
124
      port: 8585
125
      targetPort: 8585
126
    - name: "8686"
127
      port: 8686
128
      targetPort: 8686
129
---
130
apiVersion: apps/v1
131
kind: Deployment
132
metadata:
133
  labels:
134
    app: testproject-agent
135
    name: testproject-agent
136
  name: testproject-agent
137
spec:
138
  replicas: 1
139
  selector:
140
    matchLabels:
141
      app: testproject-agent
142
  template:
143
    metadata:
144
      labels:
145
        app: testproject-agent
146
        name: testproject-agent
147
    spec:
148
      restartPolicy: Always    
149
      containers:
150
        - env:
151
            - name: CHROME
152
              value: chrome:4444
153
            - name: FIREFOX
154
              value: firefox:4444
155
            - name: TP_AGENT_TEMP
156
              value: "true"
157
            - name: TP_AGENT_ALIAS # OPTIONAL variable that can be used for registering the agent with a custom alias.
158
              value: "My First k8s Agent" 
159
            - name: TP_API_KEY # The API key can be created in the TestProject Web Application (as described https://docs.testproject.io/testproject-integrations/integration-with-jenkins#creating-a-testproject-api-key).
160
              valueFrom:
161
                secretKeyRef:
162
                  name: testproject-secrets
163
                  key: TP_API_KEY  
164
              valueFrom:
165
                secretKeyRef:
166
                  name: testproject-secrets
167
                  key: TP_API_KEY             
168
            - name: TP_SDK_PORT # Agent will listen on port 8686 for SDK connections
169
              value: "8686"
170
          image: testproject/agent:latest
171
          name: testproject-agent
172
          ports:
173
            - containerPort: 8585
174
            - containerPort: 8686  
175
          livenessProbe: # The chrome pod take about ~30 sec to be ready. We need to start agent registration ONLY when booth Firefox and Chrome is available. 
176
            exec:
177
              command:
178
              - /bin/sh
179
              - -c              
180
              - curl -sSL http://chrome:4444/wd/hub/status | jq -r '.value.ready' | grep "true" || exit 1              
181
            initialDelaySeconds: 30
182
            periodSeconds: 5
183
   
Copied!
Running multiple agents
The following will bring up an additional agent named My First k8s Agent-2
1
cat k8s-testproject.yaml  | sed "s/My First k8s Agent/My First k8s Agent-2/g" | sed "s/testproject-agent/testproject-agent-2/g" | kubectl apply -f - ;
Copied!
As result, You will have 2 TestProject Agents:

The first one from the firstly run with default parameters bring up the latest Firefox and Chrome
The second brings up an additional agent named  My First k8s Agent-2 that work against the same Firefox and Chrome as My First k8s Agent 
Test different browser versions
The following  will bring up an additional TestProject agent named My First k8s Agent-3
Firefox with version standalone-firefox:86.0
1
cat k8s-testproject.yaml  | sed "s/My First k8s Agent/My First k8s Agent-3/g" | sed "s/testproject-agent/testproject-agent-3/g" |  sed "s/name: firefox/name: firefox-3/g" | sed "s/selenium\/standalone-firefox/selenium\/standalone-firefox:86.0/g" | kubectl apply -f - ;
Copied!
As result, you will have 3 TestProject Agents:

the first one My First k8s Agent from the firstly run with default parameters and the latest Firefox and Chrome
the second My First k8s Agent-2 running with default parameters and the latest Firefox and Chrome
the third My First k8s Agent-3 that work against the latest Chrome but Firefox:86.0
Flow
​
​

TestProject Agents - Previous

TestProject Agent in Docker

Next - TestProject Agents

TestProject Agent CLI

Last modified 8mo ago

Copy link

Contents

Running multiple agents

Test different browser versions

Flow