Azure Service Bus Queue - QueueClient.Receive() returning null BrokeredMessage when messages are in the queue

Question

I have messages in an Azure Service Bus Queue that I can't receive. And I'm not getting any indicator as to what the problem is. I think it has something to do with message size. You can see from the code below that I'm using OpenFileDialog. I'm selecting jpeg images and they are getting sent to the queue. Now, when I send small images less than about 50KB, they are getting displayed fine by the receiving process, but larger ones over 100KB are just staying in the queue. MSDN says that message size limit is 256KB so I'm not sure what is going on here.

I have two classes. One is SendToQueue and the other is RecvFromQueue. Here is the code.

using System;
using System.IO;
using System.Windows.Forms;
using Microsoft.ServiceBus;
using Microsoft.ServiceBus.Messaging;
using Microsoft.WindowsAzure;

namespace ServiceBusQueueApp
{
    public class SendToQueue
    {
        private const string c_testqueue = "TestQueue";

        [STAThreadAttribute]
        static void Main(string[] args)
        {
            // Configure Queue Settings
            QueueDescription qd = new QueueDescription(c_testqueue)
            {
                MaxSizeInMegabytes = 5120,
                DefaultMessageTimeToLive = new TimeSpan(1, 1, 0)
            };

            // Create a new Queue with custom settings
            string connectionString =
                CloudConfigurationManager.GetSetting("Microsoft.ServiceBus.ConnectionString");

            var namespaceManager =
                NamespaceManager.CreateFromConnectionString(connectionString);

            if (!namespaceManager.QueueExists(c_testqueue))
            {
                namespaceManager.CreateQueue(qd);
            }

            namespaceManager.DeleteQueue(qd.Path);
            namespaceManager.CreateQueue(qd);

            QueueClient client = QueueClient.CreateFromConnectionString(connectionString, c_testqueue);

            double maxSize = Math.Pow(2, 18);

            OpenFileDialog openFile = new OpenFileDialog();
            while (true)
            {
                if (openFile.ShowDialog() == DialogResult.Cancel)
                {
                    break;
                }

                var messageBodyStream = new FileStream(openFile.FileName, System.IO.FileMode.Open, FileAccess.Read, FileShare.ReadWrite);

                if (messageBodyStream.Length > maxSize)
                {
                    MessageBox.Show("File is larger than 256KB.");
                    continue;
                }
                BrokeredMessage msg =
                    new BrokeredMessage(messageBodyStream);
                msg.Properties["MyProperty"] = "Test Value";


                try
                {
                    //send msg to the queue
                    client.Send(msg);
                }
                catch (Exception exception)
                {
                    MessageBox.Show(exception.Message);
                    throw;
                }
            }

        }
    }
}


using System;
using System.Diagnostics;
using System.IO;
using System.Threading;
using System.Windows.Forms;
using Microsoft.ServiceBus;
using Microsoft.ServiceBus.Messaging;
using Microsoft.WindowsAzure;

namespace ServiceBusQueueApp
{
    class RecvFromQueue
    {

        private const string c_testqueue = "TestQueue";

        static void Main(string[] args)
        {
            // Create a new Queue with custom settings
            string connectionString =
                CloudConfigurationManager.GetSetting("Microsoft.ServiceBus.ConnectionString");

            var namespaceManager =
                NamespaceManager.CreateFromConnectionString(connectionString);

            if (!namespaceManager.QueueExists(c_testqueue))
            {
                MessageBox.Show("Queue does not exist.");
                throw new Exception("Queue does not exist.");
            }

            QueueClient client = QueueClient.CreateFromConnectionString(connectionString, c_testqueue);

            while (true)
            {
                BrokeredMessage message = client.Receive();

                if (message == null)
                {
                    continue;
                }
                try
                {
                    Stream fstream = message.GetBody<Stream>();
                    byte[] buffer = new byte[fstream.Length];
                    fstream.Read(buffer, 0, (int)fstream.Length);
                    File.WriteAllBytes(@"C:\users\roberthar\pictures\testpic.png", buffer);
                    fstream.Close();

                    Process paint = new Process();
                    paint.StartInfo.FileName = @"C:\Windows\System32\mspaint.exe";
                    paint.StartInfo.Arguments = @"C:\users\roberthar\pictures\testpic.png";
                    paint.Start();

                    Thread.Sleep(3000);

                    paint.Close();

                    // Remove message from queue
                    message.Complete();
                }
                catch (Exception exception)
                {
                    // Indicate a problem, unlock message in queue
                    message.Abandon();
                }
            }
        }
    }
}

Answer 1

Message size limit is 256 KB but this includes both the headers and the body, where the maximum header size is 64 KB. In your case header is not an issue (less than 1 KB)

I run your example with few minor changes:

            string filePath = Path.Combine(AppDomain.CurrentDomain.BaseDirectory,
                string.Format("testpic_{0}_{1}_{2}.png", now.Hour, now.Minute, now.Second));
            File.WriteAllBytes(filePath, buffer);
            fstream.Close();

            Process paint = new Process();
            paint.StartInfo.FileName = @"C:\Windows\System32\mspaint.exe";
            paint.StartInfo.Arguments = filePath;
            paint.Start();

And I was able to successfully send and receive message with this 254 KB image

If your message would be too large you would get MessageSizeExceededException when you call client.Send(msg);

You can run this test on your queue to see if you can receive all messages, it passes for me.

    [Fact]
    public void MaxMessageSize()
    {
        var sender = CreateClient();
        var reciver = CreateClient();
        for (int i = 1; i < 255; i++)
        {
            var size = i*1024;
            var buffer = new byte[size];
            random.NextBytes(buffer);
             BrokeredMessage msg =
               new BrokeredMessage(buffer);
            msg.Properties["size"] = size;
            sender.Send(msg);
            var message  = reciver.Receive();
            Assert.NotNull(message);
            Assert.Equal(message.Properties["size"], size);
            var bufferReceived = message.GetBody<byte[]>();
            Assert.Equal(buffer, bufferReceived);
            message.Complete();
        }
    }

Full gist here

Answer 2

I stumbled upon this question helping a co-worker. (I promise it was another dev!)

We ran into this issue while he was running code that checked the queue.MessageCount (setting it to a variable of name myQMessageCount ) and the code had "while (myQMessageCount > 0)" and he was resetting the queuecount after every msg.Complete (inside the same while loop)

Turns out .MessageCount is a "sum" of all messages in the queue, including Active (the ones you should be able to read) and dead letters and others.

So (1), the fix was for him to change his code to check the ActiveMessageCount, not the .MessageCount

                Microsoft.ServiceBus.Messaging.QueueDescription qd = myMicrosoftdotServiceBusdotNamespaceManager.GetQueue(qName);
                string deadLetterQueueName = QueueClient.FormatDeadLetterPath(qd.Path);
                int activeMessageCount = qd.MessageCountDetails.ActiveMessageCount;
                int deadLetterMessageCount = qd.MessageCountDetails.DeadLetterMessageCount;
                int scheduledMessageCount = qd.MessageCountDetails.ScheduledMessageCount;
                int transferDeadLetterMessageCount = qd.MessageCountDetails.TransferDeadLetterMessageCount;
                int transferMessageCount = qd.MessageCountDetails.TransferMessageCount;

and (2), after we discussed it, it probably isn't wise to keep checking the ActiveMessageCount, and just let a returned null BrokeredMessage be the check that there are no more messages in the queue.

Anyway. I am posting this answer here for future readers who might get stuck on some custom read-queue code they are writing.