Use CircleCI version 2.1 at the top of your .circleci/config.yml file.
1
version: 2.1
Add the orbs
stanza below your version, invoking the orb:
1
2
orbs:
sumologic: circleci/sumologic@1.3.0
Use sumologic
elements in your existing workflows and jobs.
This example workflow shows a typical build, test, and deploy job in a single workflow. By adding the Sumo Logic Orb to the top of our config and including the provided "workflow-collector" job, analytics may be automatically reported to the CircleCI App for Sumo Logic. You must also setup the CircleCI App on Sumo Logic
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
version: '2.1'
orbs:
sumologic: circleci/sumologic@x.y.z
jobs:
build:
docker:
- image: cimg/node
steps:
- run: echo "This job represents a typical build job"
deploy:
docker:
- image: cimg/node
steps:
- run: echo "This job represents a typical deploy job"
test:
docker:
- image: cimg/node
steps:
- run: echo "This job represents a typical test job"
workflows:
build-test-and-deploy:
jobs:
- sumologic/workflow-collector:
custom-data: |
{"foo": "bar", "$SOME_KEY_VAR": "$SOME_VALUE_VAR"}
- build
- test:
requires:
- build
- deploy:
requires:
- test
"Add this job to your workflow with no require statements. This job will run in parallel with the rest of your workflow for monitoring and will exit when all other jobs have completed."
PARAMETER | DESCRIPTION | REQUIRED | DEFAULT | TYPE |
---|---|---|---|---|
circle-token | Enter your CircleCI Personal Access Token for interacting with the API. You may generate one here: https://circleci.com/account/api | No | CIRCLE_TOKEN | env_var_name |
custom-data | A valid JSON object to append to the workflow data. | No | '' | string |
job-collector | Enter the HTTP Source endpoint generated by the Sumologic setup wizard for Workflows. | No | JOB_HTTP_SOURCE | env_var_name |
workflow-collector | Enter the HTTP Source endpoint generated by the Sumologic setup wizard for Workflows. | No | WORKFLOW_HTTP_SOURCE | env_var_name |
"Collect data from this job and report to SumoLogic. Note: MUST BE LAST COMMAND IN JOB."
PARAMETER | DESCRIPTION | REQUIRED | DEFAULT | TYPE |
---|---|---|---|---|
http-source | Enter the HTTP Source endpoint generated by the Sumologic setup wizard. | No | HTTP_SOURCE | env_var_name |
path | absolute or relative path to the logfile you wish to upload. | Yes | - | string |
Default reporting environment for Sumologic. This is a small Ubuntu based Docker image with a low resource class. Designed to load fast and use few credits.
PARAMETER | DESCRIPTION | REQUIRED | DEFAULT | TYPE |
---|---|---|---|---|
image | Docker image name | No | cimg/base | string |
tag | Docker image tag | No | stable | string |
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
# This code is licensed from CircleCI to the user under the MIT license.
# See here for details: https://circleci.com/developer/orbs/licensing
version: 2.1
description: |
Report CircleCI job analytics to your SumoLogic dashboard.
display:
home_url: https://www.sumologic.com/
source_url: https://github.com/CircleCI-Public/sumologic-orb
orbs:
jq: circleci/jq@2.2
commands:
submit:
description: |
"Collect data from this job and report to SumoLogic. Note: MUST BE LAST COMMAND IN JOB."
parameters:
http-source:
default: HTTP_SOURCE
description: Enter the HTTP Source endpoint generated by the Sumologic setup wizard.
type: env_var_name
path:
description: absolute or relative path to the logfile you wish to upload.
type: string
steps:
- run:
command: |
# Send data to SumoLogic
curl -s -X POST -T << parameters.path >> $<< parameters.http-source >>
name: SumoLogic Send Log
when: always
executors:
default:
description: |
Default reporting environment for Sumologic. This is a small Ubuntu based Docker image with a low resource class. Designed to load fast and use few credits.
docker:
- image: <<parameters.image>>:<<parameters.tag>>
parameters:
image:
default: cimg/base
description: Docker image name
type: string
tag:
default: stable
description: Docker image tag
type: string
resource_class: small
jobs:
workflow-collector:
description: |
"Add this job to your workflow with no require statements. This job will run in parallel with the rest of your workflow for monitoring and will exit when all other jobs have completed."
executor: default
parameters:
circle-token:
default: CIRCLE_TOKEN
description: 'Enter your CircleCI Personal Access Token for interacting with the API. You may generate one here: https://circleci.com/account/api'
type: env_var_name
custom-data:
default: ""
description: A valid JSON object to append to the workflow data.
type: string
job-collector:
default: JOB_HTTP_SOURCE
description: Enter the HTTP Source endpoint generated by the Sumologic setup wizard for Workflows.
type: env_var_name
workflow-collector:
default: WORKFLOW_HTTP_SOURCE
description: Enter the HTTP Source endpoint generated by the Sumologic setup wizard for Workflows.
type: env_var_name
steps:
- jq/install
- run:
command: |
###############
# Begin Collecting
###############
DATA_URL="https://circleci.com/api/v2/workflow/$CIRCLE_WORKFLOW_ID/job?circle-token=$<< parameters.circle-token >>"
WF_DATA=$(curl -s "$DATA_URL")
WF_MESSAGE=$(echo "$WF_DATA" | jq '.message')
# Exit if no Workflow.
if [ "$WF_MESSAGE" = "\"Workflow not found\"" ];
then
echo "No Workflow was found."
echo "Your circle-token parameter may be wrong or you do not have access to this Workflow."
exit 1
fi
WF_ITEMS=$(echo "$WF_DATA" | jq '.items')
WF_LENGTH=$(echo "$WF_ITEMS" | jq length)
# GET URL PATH DATA
VCS_SHORT=$(echo $CIRCLE_BUILD_URL | cut -d"/" -f4)
case $VCS_SHORT in
gh)
VCS=github
;;
bb)
VCS=bitbucket
;;
*)
echo "No VCS found. Error" && exit 1
;;
esac
# Assume the WF is currently running
WF_FINISHED=false
echo -e "Jobs in Workflow: $WF_LENGTH \n"
# Exit if no other jobs in the Workflow.
if [ "$WF_LENGTH" -lt 2 ];
then
echo "Only a single job has been found in the workflow, indicating this reporter is the only job in the pipeline."
echo "Please add other jobs to the Workflow you wish to collect data on to send to Sumologic"
exit 0
fi
#####################
## START MAIN LOOP ##
#####################
# Check the status of all jobs in the workflow that are not this job and wait until they have all finished.
while [ "$WF_FINISHED" = false ]
do
WF_DATA=$(curl -s "$DATA_URL" | jq '.items')
echo "Waiting for other jobs to finish..."
#####
# Send Workflow Data to SumoLogic
#####
mkdir -p /tmp/sumologic-logs/
WF_SL_PAYLOAD=$(curl -s "$DATA_URL" | jq '.')
echo $WF_SL_PAYLOAD > /tmp/sumologic-logs/workflow-collector.json
#curl -s -X POST -T /tmp/sumologic-logs/workflow-collector.json $<< parameters.workflow-collector >>
# for each job in the workflow fetch the status.
# the WF_FINISHED will be assumed true unless one of the jobs in the Workflow is still running
# the flag will then be set back to false.
WF_FINISHED=true
i="0"
################
### JOB LOOP ###
################
while [ $i -lt "$WF_LENGTH" ]
do
echo "looping: $i"
# fetch the job info
JOB_DATA=$(echo "$WF_DATA" | jq --arg i "$i" ".[$i]")
JOB_NUMBER=$(echo "$JOB_DATA" | jq ".job_number")
JOB_STATUS=$(echo "$JOB_DATA" | jq ".status")
JOB_NAME=$(echo "$JOB_DATA" | jq ".name")
# Only check the job if it is not this current job
if [ "$JOB_NUMBER" = "$CIRCLE_BUILD_NUM" ];
then
echo "This is the reporter job. Skipping"
else
# If this job is NOT the current job, check the status
echo "JOB: $JOB_NAME"
echo "JOB NUM: $JOB_NUMBER"
echo "STATUS: $JOB_STATUS"
### This section could be used to send data from the job while it is running. Commenting out this section currently to reduce complexity of dashboard. Updates will be sent when the job completes.
# #####
# # Send Job Data to SumoLogic
# #####
# mkdir -p /tmp/sumologic-logs/
# ###
# # Process Job api 1.1 Data
# ###
# echo
# echo "JOB DATA URL:"
# echo "https://circleci.com/api/v1.1/project/$VCS/$CIRCLE_PROJECT_USERNAME/$CIRCLE_PROJECT_REPONAME/$JOB_NUMBER"
# JOB_DATA_RAW=$(curl -s "https://circleci.com/api/v1.1/project/$VCS/$CIRCLE_PROJECT_USERNAME/$CIRCLE_PROJECT_REPONAME/$JOB_NUMBER?circle-token=$<< parameters.circle-token >>")
# # removing steps,circle_yml, outcome keys from object while the workflow is incomplete.
# JOB_DATA_RAW=$(echo $JOB_DATA_RAW | jq 'del(.circle_yml)' | jq 'del(.steps)')
# # Write the modified data to a file
# echo $JOB_DATA_RAW > /tmp/sumologic-logs/job-collector.json
# curl -s -X POST -T /tmp/sumologic-logs/job-collector.json $<< parameters.job-collector >>
###
if [ "$JOB_STATUS" == '"success"' ] || [ "$JOB_STATUS" == '"failed"' ];
then
echo "Job $JOB_NAME $JOB_NUMBER is complete - $JOB_STATUS"
elif [ "$JOB_STATUS" == '"on_hold"' ] || [ "$JOB_STATUS" == '"blocked"' ];
then
# The condition to not block metrics sending when workflow use manually approved steps or is blocked.
echo "Job $JOB_NAME $JOB_NUMBER need manual approval - $JOB_STATUS - skipping"
else
# If it is still running, then mark WF_FINISHED as false.
WF_FINISHED=false
echo "Setting status of WF_FINISHED to false"
fi
fi
echo "rerunning loop"
i="$((i+1))"
echo "increment loop to $i"
echo " ---------- "
echo
done
echo "Waiting 10 seconds"
sleep 10
done
echo
################
# WF COMPLETE #
################
echo
echo "-------------------------------"
echo "All jobs in Workflow complete."
echo "Sending final report."
echo "-------------------------------"
echo
echo
########################################
# Send end-of-workflow data to Sumologic
########################################
WF_SL_PAYLOAD=$(curl -s "https://circleci.com/api/v2/workflow/$CIRCLE_WORKFLOW_ID?circle-token=$<< parameters.circle-token >>" | jq '.')
# Append any custom data to the workflow data
ESCAPED_JSON=$(echo '<< parameters.custom-data >>' | sed -E 's/([^\]|^)"/\1\\"/g')
CUSTOM_DATA=$(eval "echo $ESCAPED_JSON")
if [[ ! -z '<< parameters.custom-data >>' ]] && echo "$CUSTOM_DATA" | jq -e;
then
echo "Appending custom data to the workflow data"
WF_SL_PAYLOAD=$(echo $WF_SL_PAYLOAD | jq -c ". + {\"custom_data\": $CUSTOM_DATA} + {\"items\": $WF_ITEMS}")
else
echo "No valid custom data found to append to the workflow data"
fi
echo "SENDING FINAL WORKFLOW DATA"
echo $WF_SL_PAYLOAD
echo $WF_SL_PAYLOAD > /tmp/sumologic-logs/workflow-collector.json
curl -s -X POST -T /tmp/sumologic-logs/workflow-collector.json $<< parameters.workflow-collector >>
########################################
# Send end-of-workflow jobs data to Sumologic
########################################
echo "SENDING FINAL JOB INFORMATION"
i=0
while [ $i -lt "$WF_LENGTH" ]
do
echo "looping: $i"
# fetch the job info
JOB_DATA=$(echo "$WF_DATA" | jq --arg i "$i" ".[$i]")
JOB_NUMBER=$(echo "$JOB_DATA" | jq ".job_number")
JOB_STATUS=$(echo "$JOB_DATA" | jq ".status")
JOB_NAME=$(echo "$JOB_DATA" | jq ".name")
# Only check the job if it is not this current job
if [ "$JOB_NUMBER" = "$CIRCLE_BUILD_NUM" ];
then
echo "This is the reporter job. Skipping"
else
# If this job is NOT the current job, check the status
echo "JOB: $JOB_NAME"
echo "JOB NUM: $JOB_NUMBER"
echo "STATUS: $JOB_STATUS"
#####
# Send Job Data to SumoLogic
#####
mkdir -p /tmp/sumologic-logs/
###
# Process Job api 1.1 Data
###
# If $JOB_NUMBER is null, probably Approval Job. Don't process and send.
if [ "$JOB_NUMBER" = "null" ];
then
echo "Approval Job, skipping"
else
JOB_DATA_RAW=$(curl -s "https://circleci.com/api/v1.1/project/$VCS/$CIRCLE_PROJECT_USERNAME/$CIRCLE_PROJECT_REPONAME/$JOB_NUMBER?circle-token=$<< parameters.circle-token >>")
# removing steps and circle_yml keys from object
JOB_DATA_RAW=$(echo $JOB_DATA_RAW | jq 'del(.circle_yml)' | jq 'del(.steps)')
# manually set job name as it is currently null
JOB_DATA_RAW=$(echo $JOB_DATA_RAW | jq --arg JOBNAME "$JOB_NAME" '.job_name = $JOBNAME')
# Append any custom data to the job data
if [[ ! -z '<< parameters.custom-data >>' ]] && echo "$CUSTOM_DATA" | jq -e;
then
echo "Appending custom data to the workflow data"
JOB_DATA_RAW=$(echo $JOB_DATA_RAW | jq -c ". + {\"custom_data\": $CUSTOM_DATA}")
else
echo "No valid custom data found to append to the job data"
fi
# Write the modified data to a file
echo $JOB_DATA_RAW > /tmp/sumologic-logs/job-collector.json
curl -s -X POST -T /tmp/sumologic-logs/job-collector.json $<< parameters.job-collector >>
###
fi
fi
echo "rerunning loop"
i="$((i+1))"
echo "increment loop to $i"
echo " ---------- "
echo
done
echo "Complete. You may now find your logs on Sumologic."
name: Workflow Collector
examples:
workflow_collector:
description: |
This example workflow shows a typical build, test, and deploy job in a single workflow. By adding the Sumo Logic Orb to the top of our config and including the provided "workflow-collector" job, analytics may be automatically reported to the CircleCI App for Sumo Logic. You must also setup the CircleCI App on Sumo Logic
usage:
version: "2.1"
orbs:
sumologic: circleci/sumologic@x.y.z
jobs:
build:
docker:
- image: cimg/node
steps:
- run: echo "This job represents a typical build job"
deploy:
docker:
- image: cimg/node
steps:
- run: echo "This job represents a typical deploy job"
test:
docker:
- image: cimg/node
steps:
- run: echo "This job represents a typical test job"
workflows:
build-test-and-deploy:
jobs:
- sumologic/workflow-collector:
custom-data: |
{"foo": "bar", "$SOME_KEY_VAR": "$SOME_VALUE_VAR"}
- build
- test:
requires:
- build
- deploy:
requires:
- test