How to generate more random map in Unity?

Question

I want to generate 2D map with landscape for my game. Now I am using Perlin noise to do that.

private List<List<int>> GenerateTerrain()
{
    var result = new List<List<int>>();
    for (var x = 0; x < mapSize; x++)
    {
        var row = new List<int>();
        for (var y = 0; y < mapSize; y++)
        {
            row.Add(GenerateNoise(x, y));
        }
        result.Add(row);
    }
    return result;
}

private int GenerateNoise(int x, int y)
{
    var rawNoise = Mathf.PerlinNoise((x - noiseOffsetX) / noiseScale, (y - noiseOffsetY) / noiseScale);
    var clampedNoise = Mathf.Clamp(rawNoise, 0, 1);
    if (clampedNoise > 0.85)
    {
        return 4;
    }
    else if (clampedNoise > 0.75)
    {
        return 3;
    }
    else if (clampedNoise > 0.60)
    {
        return 2;
    }
    else if (clampedNoise > 0.35)
    {
        return 1;
    }
    else
    {
        return 0;
    }
}

Here in GenerateNoise imagine that 0 is the lowest height of the map (bottom of the ocean for example) and 4 is the highest (mountain).

The problem here is that map generates "evenly" - if there is a mountain (4) it will be sorrounded by lower mountains (3) and they will be surrounded by plains (2) and etc. But I want more random generation - cliffs with growing height (from 0 to 4) from the east, for example, and then immediately dropping to 0 on west; or just mountain range (4) in ocean (0).

Is it possible to do? Maybe there is some another type of noise which can do that?

I have an idea of preparing some structures somewhere and then with some chance generating them. But the problem here is that I need to prepare a lot of structures, especially for mountain ranges. And I still don't fully know how to properly integrate them in terrain after base generation.