StaggeredGridLayoutManager strange effect when list scrolled

Question

I have to implement a list of images like pinterest's one (a grid with asymmetrical rows), and i have used StaggeredGridLayoutManager in this way in my fragment:

ScenarioFragment

scenarioAdapter = Adapter(mutableListOf()) {
            with(requireActivity() as MainActivity) {
                startImageActivity(it.imageLink, it.name)
            }
        }
        scenarioAdapter.clearList()
        val mLayoutManager = StaggeredGridLayoutManager(2, StaggeredGridLayoutManager.VERTICAL)
        mLayoutManager.gapStrategy = StaggeredGridLayoutManager.GAP_HANDLING_NONE
        scenario_list.layoutManager = mLayoutManager
        scenario_list.adapter = scenarioAdapter

        scenarioViewModel.getScenario()

Since this list of images is paginated, in my onViewCreated i call this metod:

private fun initScrollListener() {
        scenario_list.addOnScrollListener(object : RecyclerView.OnScrollListener() {
            override fun onScrolled(recyclerView: RecyclerView, dx: Int, dy: Int) {
                super.onScrolled(recyclerView, dx, dy)
                val layoutManager = recyclerView.layoutManager as StaggeredGridLayoutManager
                layoutManager.invalidateSpanAssignments()
                val lastVisibleItemPositions = intArrayOf(0,0)
                layoutManager.findLastVisibleItemPositions(lastVisibleItemPositions)
                if ((lastVisibleItemPositions[1] == scenarioAdapter.itemCount - 1) &&
                    scenarioAdapter.itemCount >= BuildConfig.PAGE_SIZE && keepScrolling && !alreadyRefreshed) {
                    alreadyRefreshed = true
                    scenarioViewModel.getScenario(scenarioAdapter.getPage())
                }
            }
        })
    }

and it works fine, it load the images correctly.

This is the Adapter

 class Adapter(
        private val items: MutableList<ScenarioModel>,
        private val click: (ScenarioModel) -> Unit
    ) :
        RecyclerView.Adapter<Adapter.ViewHolder>() {

        fun update(list: MutableList<ScenarioModel>) {
            this.items.addAll(list)
            notifyDataSetChanged()
        }

        fun clearList() = items.clear()

        fun getPage(): Int = (items.size / BuildConfig.PAGE_SIZE)

        override fun onCreateViewHolder(
            parent: ViewGroup,
            viewType: Int
        ): ViewHolder {
            val view =
                LayoutInflater.from(parent.context)
                    .inflate(R.layout.item_scenario_dynamic, parent, false)
            return ViewHolder(view)
        }

        override fun onBindViewHolder(holder: ViewHolder, position: Int) {
            holder.bindItem(items[position], click)
        }

        override fun getItemCount(): Int = items.size

        class ViewHolder(view: View) : RecyclerView.ViewHolder(view) {
            fun bindItem(item: ScenarioModel, click: (ScenarioModel) -> Unit) {
                GlideApp.with(itemView.context).load(item.imageLink).into(itemView.scenario_image)
                itemView.setOnClickListener { click(item) }
            }
        }
    }

The problem is this: when the fragment is displayed, i have 10 items shown. When i scroll down, it works perfectly. But when i go back, scrolling up, the items do an animation like if they are reordered or reloaded, it changes every time. Who can I fix this?

PS: the problem is not associated with the pagination, I tried to get only the first page, with a big size, but the problem still.