Office365-REST-Python-Client Connected but no items showing

Question

I'm trying to access sharepoint with Office365-Rest. I'm using the app connection process described here:

https://learn.microsoft.com/en-us/sharepoint/dev/solution-guidance/security-apponly-azureacs

I set permissions to

<AppPermissionRequests AllowAppOnlyPolicy="true">
  <AppPermissionRequest Scope=" https://*.sharepoint.com/sites/Engineering/content/tenant" Right="FullControl" />
</AppPermissionRequests>

Python Code:

import sys
from office365.runtime.auth.client_credential import ClientCredential
from office365.runtime.http.request_options import RequestOptions
from office365.sharepoint.client_context import ClientContext


client_id="*"
client_secret="*"

sp='https://*.sharepoint.com/sites/Engineering/'
client_credentials = ClientCredential(client_id,client_secret)

web = ctx.web.get().execute_query()
print(web.url)

ctx = ClientContext(sp).with_credentials(client_credentials)
list_source = ctx.web.lists.get_by_title("PDF")

items = mylist.get_items().select("ID").top(10)
ctx.execute_query()
for i in items:
    print(i)

My connection goes through. I get the web.url printed. If I change the folder or access codes, I get errors. However whatever I seem to try and get data from lists or folders, I never get any data.

I've run out of ideas on how to test or what to do next, any help would be appreciated.

I'm trying to access 3 items, 1 folder with 5000+ item, 1 list with 5000+ items and 1 folder with a 100 items. In all cases of trying to access these, I'm not getting anything. I thought with the folder of 100 items, I should get something and then I could work on developing code to go through the big lists. I've tried the following example code from the github to no avail.

https://github.com/vgrem/Office365-REST-Python-Client/blob/55252191cb7f729f044f61542dca309249c743fc/examples/sharepoint/lists_and_items/read_large_list.py

I managed to use shareplum to update lists, but that doesn't seem to currently have a workaround for 5000+ lists so I was hoping this would do it once I get the data working.

Answer 1

A SharePoint list that is in Default mode will block certain lookup calls when the overall list count is greater than 5000 rows. You must set it to the New experience mode for large lists.

For large lists when performing queries that would return a very small number of items will fail with no failure code or exception if the columns used to filter the query are not indexed. In the my case the filter is "(Title eq 'filename') and (FindingSource eq 'Automated')". Indexing both of those columns solved the problem.