Passing custom flags to "open" in a device driver

Question

I need to pass some custom flags to the open() call of my device driver.

I found this example in LDD3:

int dev_open(struct inode *inode, struct file *filp)
{
    if ((filp->f_flags & O_ACCMODE) == O_WRONLY) {
        ...
    }
}

My question is: is it possibile to define other flags (like O_ACCMODE and O_WRONLY) without conflicts with any others?

Answer 1

Yes, it's possible. Take a look at include/uapi/asm-generic/fcntl.h. Pay attention to next comment:

/*
 * When introducing new O_* bits, please check its uniqueness in fcntl_init().
 */

Now look into fcntl_init() function (defined at fs/fcntl.c):

/*
 * Please add new bits here to ensure allocation uniqueness.
 * Exceptions: O_NONBLOCK is a two bit define on parisc; O_NDELAY
 * is defined as O_NONBLOCK on some platforms and not on others.
 */
BUILD_BUG_ON(20 - 1 /* for O_RDONLY being 0 */ != HWEIGHT32(
    O_RDONLY        | O_WRONLY      | O_RDWR        |
    O_CREAT         | O_EXCL        | O_NOCTTY      |
    O_TRUNC         | O_APPEND      | /* O_NONBLOCK | */
    __O_SYNC        | O_DSYNC       | FASYNC        |
    O_DIRECT        | O_LARGEFILE   | O_DIRECTORY   |
    O_NOFOLLOW      | O_NOATIME     | O_CLOEXEC     |
    __FMODE_EXEC    | O_PATH        | __O_TMPFILE
    ));

So first you need to find unique value for your new definition, so it can be bitwise-or'd with flags listed in fcntl_init(). Next you need to add your new definition to include/uapi/asm-generic/fcntl.h. And finally add your new define to fcntl_init(), so it will be checked at compile time.

In the end it boils down to finding the value that doesn't conflict with existing definitions. E.g. as I can see all 10, 100, 1000, 10000, 100000, 1000000 and 10000000 are used. So for your new flags you can use 100000000, 200000000, 400000000 and 800000000 values.

UPDATE: As SailorCaire correctly mentioned, you also need to increment first number in BUILD_BUG_ON() macro. For example, if it originally was BUILD_BUG_ON(20 - 1, and you are to add one element to this list, you should make it BUILD_BUG_ON(21 - 1.

UPDATE 2: Another valuable addition from SailorCaire:

By the way, you'll need to do make install_headers, copy the new headers, and it looks like you'll need to recompile glibc so it becomes aware of the API change.