Pymongo: Best way to remove $oid in Response

Question

I have started using Pymongo recently and now I want to find the best way to remove $oid in Response

When I use find:

result = db.nodes.find_one({ "name": "Archer" }

And get the response:

json.loads(dumps(result))

The result would be:

{
  "_id": {
  "$oid": "5e7511c45cb29ef48b8cfcff"
  },
  "about": "A jazz pianist falls for an aspiring actress in Los Angeles."
}

My expected:

{
  "_id": "5e7511c45cb29ef48b8cfcff",
  "about": "A jazz pianist falls for an aspiring actress in Los Angeles."
}

As you seen, we can use:

resp = json.loads(dumps(result))
resp['id'] = resp['id']['$oid']

But I think this is not the best way. Hope you guys have better solution.

Answer 1

You can take advantage of aggregation:

result = db.nodes.aggregate([{'$match': {"name": "Archer"}}
                             {'$addFields': {"Id": '$_id.oid'}},
                             {'$project': {'_id': 0}}])
data = json.dumps(list(result))

Here, with $addFields I add a new field Id in which I introduce the value of oid. Then I make a projection where I eliminate the _id field of the result. After, as I get a cursor, I turn it into a list.

It may not work as you hope but the general idea is there.

Answer 2

Pass your result to below function. (Tested at 5 level of nesting with $oid and $data)

def remove_special_fields(d):
    if isinstance(d, dict):
        new_dict = {}
        for key, value in d.items():
            if isinstance(value, dict) and ('$oid' in value or '$date' in value):
                new_dict[key] = value.get('$oid', value.get('$date'))
            else:
                new_dict[key] = remove_special_fields(value)
        return new_dict
    elif isinstance(d, list):
        return [remove_special_fields(item) for item in d]
    else:
        return d

Answer 3

import re

def remove_oid(string):
    while True:
        pattern = re.compile('{\s*"\$oid":\s*(\"[a-z0-9]{1,}\")\s*}')
        match = re.search(pattern, string)
        if match:
            string = string.replace(match.group(0), match.group(1))
        else:
            return string

string = json_dumps(mongo_query_result)
string = remove_oid(string)

Answer 4

I am using some form of custom handler. I managed to remove $oid and replace it with just the id string:

# Custom Handler
def my_handler(x):
    if isinstance(x, datetime.datetime):
        return x.isoformat()
    elif isinstance(x, bson.objectid.ObjectId):
        return str(x)
    else:
        raise TypeError(x)

# parsing
def parse_json(data):
    return json.loads(json.dumps(data, default=my_handler))

result = db.nodes.aggregate([{'$match': {"name": "Archer"}}
                             {'$addFields': {"_id": '$_id'}},
                             {'$project': {'_id': 0}}])
data = parse_json(result)

Answer 5

In the second argument of find_one, you can define which fields to exclude, in the following way:

site_information = mongo.db.sites.find_one({'username': username}, {'_id': False})

This statement will exclude the '_id' field from being selected from the returned documents.

Answer 6

First of all, there's no $oid in the response. What you are seeing is the python driver represent the _id field as an ObjectId instance, and then the dumps() method represent the the ObjectId field as a string format. the $oid bit is just to let you know the field is an ObjectId should you need to use for some purpose later.

The next part of the answer depends on what exactly you are trying to achieve. Almost certainly you can acheive it using the result object without converting it to JSON.

If you just want to get rid of it altogether, you can do :

result = db.nodes.find_one({ "name": "Archer" }, {'_id': 0})
print(result)

which gives:

{"name": "Archer"}