How to - enable distributed tracing on our z/OS Connect servers (based on WebSphere liberty Profile and open-liberty)?

Question

I am writing to you today to request assistance in enabling distributed tracing in z/OS Connect and CICS while working with API requesters - from COBOL routines (in the scope of online CICS transactions) that activates APIs via the zCEE communication stubs.

I have a good understanding of z/OS, CICS, COBOL development, REXX, JCL, Java, WAS Liberty, and z/OS Connect. However, I am still not sure how to activate these kind of 'tracers' in our JVMs and systems.

after a basic consultation with IBM (opened a case), they referred us to the resource: https://github.com/WASdev/sample.opentracing.zipkintracer#readme but according to that resource we still couldn't understand the A-Z steps that will help us complete this mission successfully..

I would appreciate it if you could provide me with more technical support to help me with this task.

Installed products: API-M is the WSO2 System
ZCEE version - 3.0.3
Build Toolkit Version - 1.2 (20200512-1224)
Liberty Version - WAS FOR Z/OS 19.0.0.12
RELEASE z/OS 02.04.00 LICENSE = zNALC
CICS Transaction Server Version 5.6.0

Thank you for your time and consideration,

Sincerely,

Alon

Instrument our z/OS Connect server code to generate tracing data with the opentracing-1.0

Answer 1

Depending on the type of trace/monitoring information you are after, here are a few links to the relevant parts of the documentation that describe the configuration required.

Before I go any further though, a few notes:

1. The version of z/OS Connect you said you are using (3.0.3) is quite old and some of the monitoring features discussed in the links below had not yet been introduced. I suggest upgrading to the latest version (3.0.70) if possible :-)
2. The links below are for z/OS Connect OpenAPI 2 servers, since I assume that’s what you are using given the version number you are on. There is also a z/OS Connect OpenAPI 3 server (introduced last year) that provides metrics and statistics in a more standardised fashion. More info here: https://www.ibm.com/docs/en/zos-connect/zos-connect/3.0?topic=zos-connect-monitoring

Back to what you can do in the z/OS Connect OpenAPI 2 server...

For request level monitoring there are number options based around z/OS Connect’s interceptor framework.

• SMF 123 records capture details about each request flowing through z/OS Connect via the audit interceptor provided with z/OS Connect https://www.ibm.com/docs/en/zos-connect/zosconnect/3.0?topic=interceptors-configuring-audit-interceptor
• Using interceptors from Z based monitoring tools (like Omegamon for JVM) to monitor requests in realtime: https://community.ibm.com/community/user/ibmz-and-linuxone/blogs/nigel-williams2/2021/01/23/new-features-in-ibm-z-omegamon-for-jvm-v550-fix-pa
• Using the Z APM Connect interceptor to send data to end to end request tracking tools like Instana: https://www.ibm.com/docs/en/iooz?topic=components-tracking-zos-connect-enterprise-edition All of these options are discussed or linked to from the Monitoring page in the z/OS Connect documentation: https://www.ibm.com/docs/en/zos-connect/zosconnect/3.0?topic=monitoring

For internal trace and debugging, that’s covered in the problem resolution part of the documentation: https://www.ibm.com/docs/en/zos-connect/zosconnect/3.0?topic=resolving-problems

Finally pages 40 & 49-58 of the deployment planning guide available here: http://ibm.biz/zcee-deployment-guide discuss all of the above at a conceptual level and may be helpful to understanding the topic of monitoring and trace.

I hope that helps :-)