Deleted items affect to pagination

Question

I need iterate over big amount of users (10000k+) and on each user process some logic like checking if user match my delete condition, and if yes, then send email and delete user in database.

This is my idea of code, but I am facing to some problems. If I delete user, then pagination is affected. I tried paginate always from 0, but in case that some user is not processable then my code stuck in infinity loop. Can you help me how to solve that effectively?

@Service
class UserService(private val userRepository: UserRepository, private val emailService: EmailService) {

    fun processInactiveExternalUsersInBatches(pageSize: Int) {
        var page = 0
        var usersPage: Page<User>

        do {
            usersPage = userRepository.findUsersByType(userType = UserType.EXTERNAL_USER, PageRequest.of(page, pageSize))
            processUsers(usersPage.content)

            page++
        } while (usersPage.hasNext())
    }

    private fun processUsers(users: List<User>) {
        for (user in users) {
            if (!user.active) {
                // Send an email to the user about deactivation
                emailService.sendDeactivationEmail(user)

                // Delete the user using the repository
                userRepository.delete(user)
            }
        }
    }
}

Answer 1

you can fetch users (and you can find just the non-active users for better performance) and send email to users that are going to be deleted and after that execute a jpql query to delete those users.

Another solution : you can process users and send email to them but not delete them, then you can store them in a list and at the end of processInactiveExternalUsersInBatches you can delete them. please note that I modified your code in java but I hope you get the solution like :

    @Service
    class UserService(private val userRepository: UserRepository, private val 
    emailService: EmailService) {

    fun processInactiveExternalUsersInBatches(pageSize: Int) {
        var page = 0
        var usersPage: Page<User>

        **var deletedUsers = new ArrayList<User>();**

        do {
            usersPage = userRepository.findUsersByType(userType = 
            UserType.EXTERNAL_USER, PageRequest.of(page, pageSize))
            **var usersToBeDeleted =** processUsers(usersPage.content)
        
            **deletedUsers.addAll(usersToBeDeleted);**

            page++
        } while (usersPage.hasNext())

        **userRepository.deleteAll(deletedUsers);**
    }

    private **List<User>** fun processUsers(users: List<User>) {
        **var deletedUsers = new ArrayList<User>();**
        for (user in users) {
            if (!user.active) {
                // Send an email to the user about deactivation
                emailService.sendDeactivationEmail(user)

                // find deleted users and return them
                **deletedUsers.add(user)**
            }
        }
        **return deletedUsers;**
    }
}