get-log-events returns only a few results of many

Question

I am logging to CloudWatch from a python lambda. In the AWS console I can see several hundred log entries, but if I try to use get-log-events from either a Python Boto call, or through the AWS CLI I only ever seem to get the most recent 2 entries.

I use the following AWS CLI command:

aws logs get-log-events --log-group-name hall-ac --log-stream-name sensors

I've tried changing it to set time limits, read from head or tail, or change the limit, but nothing seems to have any effect. I thought that the above command should read up to 10000 entries, but it doesn't do that. Is there a way I can enable debugging to see why only some logs are being returned?

I can use the next-token parameter to retrieve a few more log entries, but it will take hundreds of calls to read every log entry if it only delivers 2 each time.

Answer 1

There are 2 limits in get-log-events: 1MB of data 10000 log events

So, whichever limit you hit first. I guess that you have bigger logs and thus you're hitting the 1MB limit

You can see more details in the documentation

Answer 2

adding the flag --start-from-head helped me out. I was using the nextToken to manage the pagination.

As per the docs: "If you are using nextToken in this operation, you must specify true for startFromHead ."