How to browse the message based on MsgId or CorrelId from queue in pymqi

Question

import pymqi

queue_manager = 'QM1'
channel = 'DEV.APP.SVRCONN'
host = '127.0.0.1'
port = '1414'
queue_name = 'TEST.1'
message = 'Hello from Python 2!'
conn_info = '%s(%s)' % (host, port)

First I did the PUT operation like the below code and I worked fine. I put for more 2000 messages

qmgr = pymqi.connect(queue_manager, channel, conn_info)
queue = pymqi.Queue(qmgr, queue_name)
queue.put(message)
queue.close()
qmgr.disconnect()

I tried the below code to get the first message It also worked fine. It gives me the first message. I don't know is the correct approach to get the first message.

queue = pymqi.Queue(qmgr, queue_name, pymqi.CMQC.MQOO_BROWSE)    
gmo = pymqi.GMO()
gmo.Options = pymqi.CMQC.MQGMO_WAIT | pymqi.CMQC.MQGMO_BROWSE_NEXT | pymqi.CMQC.MQGMO_NO_PROPERTIES
gmo.WaitInterval = 5000
md = pymqi.MD()
md.Format = pymqi.CMQC.MQFMT_STRING
message = queue.get(None, md, gmo)
print(message)   # b'Hello from Python 2!'
print(md.MsgId.hex())  #'414d51204e41544d333030202020202817d9ff650a742f22'
queue.close()
qmgr.disconnect()

but when I tried below code it gives me correct message but I this is not the best solution because if I have 2000 messages then it will iterate until the msgId matches.

queue = pymqi.Queue(qmgr, queue_name, pymqi.CMQC.MQOO_BROWSE)
gmo = pymqi.GMO()
gmo.Options = pymqi.CMQC.MQGMO_WAIT | pymqi.CMQC.MQGMO_BROWSE_NEXT | pymqi.CMQC.MQGMO_NO_PROPERTIES
gmo.WaitInterval = 5000
user_MsgId = '414d51204e41544d333030202020202817d9ff650a742f22'
overall_message = []
keep_running = True
while keep_running:
    md = pymqi.MD()
    md.Format = pymqi.CMQC.MQFMT_STRING
    message = queue.get(None, md, gmo)
    print(md.MsgId.hex())
    if md.MsgId.hex() != user_MsgId:
        continue:
    else:
        overall_message.append(message)
        break
queue.close()
qmgr.disconnect()

The above solution gives me correct message but when messages are more performance goes down. Can anyone please suggest better approach to browse the message by MsgId or CorrelId?

I tried below code as per suggestion I edited here but it is not working:

queue = pymqi.Queue(qmgr, queue_name, pymqi.CMQC.MQOO_BROWSE)    
gmo = pymqi.GMO()
gmo.MatchOptions = pymqi.CMQC.MQMO_MATCH_MSG_ID
gmo.Version = pymqi.CMQC.MQMO_VERSION_2
gmo.Options = pymqi.CMQC.MQGMO_WAIT | pymqi.CMQC.MQGMO_BROWSE_NEXT | pymqi.CMQC.MQGMO_NO_PROPERTIES
gmo.WaitInterval = 5000
md = pymqi.MD()
md.Version = pymqi.CMQC.MQMO_VERSION_2
md.Format = pymqi.CMQC.MQFMT_STRING
md.MsgId = b'414d51204e41544d333030202020202817d9ff650a742f22' # this is the same output which I get from above 
message = queue.get(None, md, gmo)
print(message)
print(md.MsgId.hex())
queue.close()
qmgr.disconnect()

Answer 1

You certainly can, and should, be using Get by Message Id.

I ran your example code and discovered that you were setting the MsgId as a string. The MsgId is a bytes field, so you want to change the line where you are setting the MsgId to something like the following:-

md.MsgId = bytes.fromhex('414D51204D51473120202020202020200110A66537AC0040')

You also had a couple of minor errors that I'm sure you'd notice soon enough. The constant MQMO_VERSION_2 does not exist. You need to correct the lines setting the gmo.Version and the md.Version thus:-

gmo.Version = pymqi.CMQC.MQGMO_VERSION_2
md.Version = pymqi.CMQC.MQMD_VERSION_2

And finally there is no need to set the md.Format on input to a queue.get. It is an output only field in that case.

Answer 2

Generally speaking, for queues that are operating on a correlation ID / message ID matching mechanism, you don't want their queue depths to become very large. The reason for this is exactly what you're running into when testing with large queue depths: that being performance goes down.

In your code, the IBM MQ client has to check each message in the queue to validate the message ID until you get a match, so in absolute worst case we're talking O(n) time to get a single message, where n is the length of the queue (and that's not even taking into account factors like network latency etc). If you use MQMO_MATCH_CORREL_ID or MQMO_MATCH_MSG_ID then the queue manager does this itself which is typically going to be faster, but it's still not as efficient as just getting the first message off the queue in a FIFO style system.

Also bear in mind that IBM MQ is not a database, and it shouldn't be treated as such. If you need the capability to search data based on ID and have very high speed responses, I would strongly recommend a production grade database with appropriate configurations (ie indexing on columns that are heavily used etc).

I'll edit this answer later today with some code examples showing a producer/consumer pattern with the flags you want set.