Seaweedfs Delete file succeds but existing filer still holds it

Question

We use seaweedfs 1.78
When i use grpc delete a file via filer.

curl -X DELETE http://filer1:9889/dataset/qiantao/1.txt

It return success.
Because I have 10 filer. after delete!

curl -H "Accept: application/json"   "http://filer2:9889/dataset/qiantao/?pretty=y"  |grep qiantao |grep txt 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 15723    0 15723    0     0  1917k      0 --:--:-- --:--:-- --:--:-- 2193k
      "FullPath": "/dataset/qiantao/1.txt",

If I start a new filer.
It can not got /dataset/qiantao/1.txt; It looks perfect!!!!
But in exist filers.
Filer get file info below.

curl -H "Accept: application/json"   "http://filer1:9889/dataset/qiantao/?pretty=y&limit=1"
{
  "Path": "/dataset/qiantao",
  "Entries": [
    {
      "FullPath": "/dataset/qiantao/1.txt",
      "Mtime": "2020-12-07T11:15:59+08:00",
      "Crtime": "2020-12-07T11:15:59+08:00",
      "Mode": 432,
      "Uid": 0,
      "Gid": 0,
      "Mime": "text/plain",
      "Replication": "010",
      "Collection": "",
      "TtlSec": 0,
      "UserName": "",
      "GroupNames": null,
      "SymlinkTarget": "",
      "Md5": null,
      "Extended": null,
      "chunks": [
        {
          "file_id": "4328,587fb084df9f9dbf",
          "size": 2,
          "mtime": 1607310959158810676,
          "e_tag": "c7c83966",
          "fid": {
            "volume_id": 4328,
            "file_key": 1484763268,
            "cookie": 3751779775
          }
        }
      ]
    }
  ],
  "Limit": 1,
  "LastFileName": "1.txt",
  "ShouldDisplayLoadMore": true

Get volume info below.

    {
      "Id": 4328,
      "Size": 31492542356,
      "ReplicaPlacement": {
        "SameRackCount": 0,
        "DiffRackCount": 1,
        "DiffDataCenterCount": 0
      },
      "Ttl": {
        "Count": 0,
        "Unit": 0
      },
      "Collection": "",
      "Version": 3,
      "FileCount": 111030,
      "DeleteCount": 709,
      "DeletedByteCount": 1628822733,
      "ReadOnly": false,
      "CompactRevision": 0,
      "ModifiedAtSecond": 0,
      "RemoteStorageName": "",
      "RemoteStorageKey": ""
    },

So download 4328.idx from volume server. and use see_idx lookup it.

./see_idx -dir /Users/qiantao/Documents/seaweedfs -volumeId=4328 -v=4 |grep 587fb084

key:587fb084 offset:2802901546 size:57
key:587fb084 offset:3937021600 size:4294967295

It looks like key:587fb084 is covered with new?

So How can I fix this problem to make it appear normal?

Answer 1

4294967295 is a tombstone, marking the entry has been deleted.