c# 1D-byte array to 2D-double array

Question

I'm dealing with c# concurrent-queue and multi-threading in socket-programming tcp/ip

First, I've already done with socket-programming itself. That means, I've already finished coding about client, server and stuffs about communication itself

basic structure is pipe-lined(producer-consumer problem) and now I'm doing with bit conversion

below is brief summary about my code

client-socket ->server-socket -> concurrent_queue_1(with type byte[65536],Thread_1 process this) -> concurrent_queue_2(with type double[40,3500], Thread_2 process this) -> display-data or other work(It can be gpu-work)

*(double[40,3500] can be changed to other size)

Till now,I've implemented putting_data into queue1(Thread1) and just dequeuing all(Thread2) and, its speed is about 700Mbps

The reason I used two concurrent_queue is, I want communication,and type conversion work to be processed in background regardless of main procedure about control things.

Here is the code about my own concurrent_queue with Blocking

public class BlockingConcurrentQueue<T> : IDisposable
    {
        private readonly ConcurrentQueue<T> _internalQueue;
        private AutoResetEvent _autoResetEvent;
        private long _consumed;
        private long _isAddingCompleted = 0;
        private long _produced;
        private long _sleeping;

        public BlockingConcurrentQueue()
        {
            _internalQueue = new ConcurrentQueue<T>();
            _produced = 0;
            _consumed = 0;
            _sleeping = 0;
            _autoResetEvent = new AutoResetEvent(false);
        }
        public bool IsAddingCompleted
        {
            get
            {
                return Interlocked.Read(ref _isAddingCompleted) == 1;
            }
        }
        public bool IsCompleted
        {
            get
            {
                if (Interlocked.Read(ref _isAddingCompleted) == 1 && _internalQueue.IsEmpty)
                    return true;
                else
                    return false;
            }
        }
        public void CompleteAdding()
        {
            Interlocked.Exchange(ref _isAddingCompleted, 1);
        }
        public void Dispose()
        {
            _autoResetEvent.Dispose();
        }
        public void Enqueue(T item)
        {
            _internalQueue.Enqueue(item);

            if (Interlocked.Read(ref _isAddingCompleted) == 1)
                throw new InvalidOperationException("Adding Completed.");

            Interlocked.Increment(ref _produced);
            if (Interlocked.Read(ref _sleeping) == 1)
            {
                Interlocked.Exchange(ref _sleeping, 0);
                _autoResetEvent.Set();
            }
        }
        public bool TryDequeue(out T result)
        {
            if (Interlocked.Read(ref _consumed) == Interlocked.Read(ref _produced))
            {
                Interlocked.Exchange(ref _sleeping, 1);
                _autoResetEvent.WaitOne();
            }
            if (_internalQueue.TryDequeue(out result))
            {
                Interlocked.Increment(ref _consumed);
                return true;
            }
            return false;
        }
    }

My question is here

As I mentioned above, concurrent_queue1's type is byte[65536] and 65536 bytes = 8192 double data. (40 * 3500=8192 * 17.08984375) I want merge multiple 8192 double data into form of double[40,3500](size can be changed)and enqueue to concurrent_queue2 with Thread2

It's easy to do it with naive-approach(using many complex for loop) but it's slow cuz, It copys all the data and expose to upper class or layer.

I'm searching method automatically enqueuing with matched size like foreach loop automatically iterates through 2D-array in row-major way, not yet found

Is there any fast way to merge 1D-byte array into form of 2D-double array and enqueue it?

Thanks for your help!

Answer 1

I try to understand your conversion rule, so I write this conversion code. Use Parallel to speed up the calculation.

int maxSize = 65536;
byte[] dim1Array = new byte[maxSize];
for (int i = 0; i < maxSize; ++i)
{
    dim1Array[i] = byte.Parse((i % 256).ToString());
}

int dim2Row = 40;
int dim2Column = 3500;
int byteToDoubleRatio = 8;
int toDoubleSize = maxSize / byteToDoubleRatio;
double[,] dim2Array = new double[dim2Row, dim2Column];
Parallel.For(0, toDoubleSize, i =>
{
    int row = i / dim2Column;
    int col = i % dim2Column;
    int originByteIndex = row * dim2Column * byteToDoubleRatio + col * byteToDoubleRatio;
    dim2Array[row, col] = BitConverter.ToDouble(
        dim1Array,
        originByteIndex);
});