Whitesource Bolt Azure DevOps plugin takes long time (and is too chatty)

Question

This happened since yesterday.

In the past months, I have integrated Whitesource Bolt scan (a free alternative to the popular Snyk) into our DevOps projects.

It normally took a couple of minutes to scan our packages, and we wew happy with the pipelines.

This is a typical redacted log from the pipelines

Starting: WhiteSource Bolt Scan
==============================================================================
Task         : WhiteSource Bolt
Description  : Detect security vulnerabilities, problematic open source licenses.
Version      : 21.3.2
Author       : WhiteSource
Help         : http://www.whitesourcesoftware.com
==============================================================================
Working directory is /home/vsts/work/1/s
Getting scan config data
unifiedAgent.config file created successfully at /home/vsts/work/1/s
Finished getScanConfigData
Finished archive and encryption
Starting Upload zip file to s3
Getting temp credentials
Finished to prepare scm scan request
Sending SCM scan request
Succeed to send SCM scan request
WhiteSource Support Token: 
Async Command Start: Add Build Tag
Build '4998' has following tags now: ws_support_token=ws_scan_start_time=Wed, 05 May 2021 12_32_26 GMT
Async Command End: Add Build Tag
Async Command Start: Add Build Tag
Build '4998' has following tags now: ws_support_token=
Async Command End: Add Build Tag
Finishing: WhiteSource Bolt Scan

Since yesterday, the output log exploded into the following endless debug logs, taking 30 minutes for an Angular project

Starting: WhiteSource Bolt Scan
==============================================================================
Task         : WhiteSource Bolt
Description  : Detect security vulnerabilities, problematic open source licenses.
Version      : 21.6.2
Author       : WhiteSource
Help         : http://www.whitesourcesoftware.com
==============================================================================





[CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]     resolved url in file = https://pkgs.dev.azure.com/_/_packaging/_/npm/registry/@babel/plugin-transform-template-literals/-/plugin-transform-template-literals-7.13.0.tgz
[DEBUG] [2021-07-06 08:41:49,836 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   resolved url in link = http://pkgs.dev.azure.com/@babel/plugin-transform-template-literals/7.13.0
[DEBUG] [2021-07-06 08:41:49,918 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   npm.accessToken is not defined
[DEBUG] [2021-07-06 08:41:50,043 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   Succeed to download the npm package @babel/plugin-transform-modules-umd-7.13.0.tgz-7.13.0.
[DEBUG] [2021-07-06 08:41:50,043 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   resolved url in file = https://pkgs.dev.azure.com/_/_packaging/_/npm/registry/@babel/plugin-transform-modules-amd/-/plugin-transform-modules-amd-7.13.0.tgz
[DEBUG] [2021-07-06 08:41:50,043 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   resolved url in link = http://pkgs.dev.azure.com/@babel/plugin-transform-modules-amd/7.13.0
[DEBUG] [2021-07-06 08:41:50,085 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   npm.accessToken is not defined
[DEBUG] [2021-07-06 08:41:50,085 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   Succeed to download the npm package @babel/plugin-syntax-optional-chaining-7.8.3.tgz-7.8.3.
[DEBUG] [2021-07-06 08:41:50,086 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   resolved url in file = https://pkgs.dev.azure.com/_/_packaging/_/npm/registry/babel-plugin-dynamic-import-node/-/babel-plugin-dynamic-import-node-2.3.3.tgz
[DEBUG] [2021-07-06 08:41:50,086 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   resolved url in link = http://pkgs.dev.azure.com/babel-plugin-dynamic-import-node/2.3.3
[DEBUG] [2021-07-06 08:41:50,146 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   npm.accessToken is not defined
[DEBUG] [2021-07-06 08:41:50,147 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   Succeed to download the npm package @babel/compat-data-7.13.8.tgz-7.13.8.
[DEBUG] [2021-07-06 08:41:50,147 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   resolved url in file = https://registry.npmjs.org/object.assign/-/object.assign-4.1.0.tgz
[DEBUG] [2021-07-06 08:41:50,147 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   resolved url in link = http://registry.npmjs.org/object.assign/4.1.0
[DEBUG] [2021-07-06 08:41:50,256 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   npm.accessToken is not defined
[DEBUG] [2021-07-06 08:41:50,258 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   Succeed to download the npm package @babel/plugin-proposal-logical-assignment-operators-7.13.8.tgz-7.13.8.
[DEBUG] [2021-07-06 08:41:50,258 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   resolved url in file = https://pkgs.dev.azure.com/_/_packaging/_/npm/registry/@babel/plugin-transform-parameters/-/plugin-transform-parameters-7.13.0.tgz
[DEBUG] [2021-07-06 08:41:50,258 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   resolved url in link = http://pkgs.dev.azure.com/@babel/plugin-transform-parameters/7.13.0
[DEBUG] [2021-07-06 08:41:51,633 +0000] - [CTX=sjgpwi107sx5to1j1pxbeasjdlvfsjqhzf6oqzeo1phtb]   npm.accessToken is not defined

We never changed the pipeline configuration

      - task: WhiteSource@21
        displayName: WhiteSource Bolt Scan
        inputs:
          cwd: '$(System.DefaultWorkingDirectory)'
          projectName: '$(projectName)'

Anyone noticed this too? What could we possibly do other than giving up on this plugin for another service?

Answer 1

This is an official feedback from Whitesource support

Starting release 21.6.2, the WhiteSource scan is performed directly within the Azure DevOps pipeline. This means that the WhiteSource task is running a scan as part of your pipeline build.

Prior to this change, the WhiteSource task didn't perform the scan directly, it collected the relevant information, sent it to a remote WhiteSource server which was the one running the scan itself. Only when the scan on the remote server finished and the results were sent back, the WhiteSource risk report on Azure DevOps was displayed. This caused the WhiteSource report to be loaded after a long time and had several issues. Thus we decide to make the changes to a direct scan and this is a much more straightforward method for scanning, and the WhiteSource report is loaded much faster and many other improvements as well. However, it is important to understand that now that the scan is performed synchronously as part of the build (and not remotely asynchronously), the build time (and not the scan time) is increased compared to the previous releases.

So it looks like they pushed a major change without warning the users that the pipeline will take longer