Instituto Superior de Engenharia do Porto

Mestrado em Engenharia Eletrotécnica e de Computadores
Arquitetura de Computadores
Loadable Kernel Module – Blocking operations and waitqueues
Deliverables: final implementation of chrdev.c to jes@isep.ipp.pt, using the following text
as subject: ARCOM-LAB7-FINAL
Deliverables: handwritten implementation of 7.2 (arcom_chrdev_read)
The minimal skeleton
Recall the minimal structure of a Linux kernel module:
Automatically allocated major number and multiple minor numbers
The module presented below creates NDEVICES character devices, all sharing the same major
number. Each character device is associated to a different minor number. The major and
minor numbers are automatically assigned by the system (alloc_chrdev_region is used
instead of register_chrdev_region), i.e., they are not “hard coded”.
Since the major and minor numbers are automatically assigned, how does the user space
processes get to know this information and create the respective device files? In this case, the
device files are created by the module itself (see the call to the device_create function),
using a denomination that is defined in module code and, therefore, is known a priori.
Additionally, device_create passes the major and minor number to sysfs1, and this
information can be queried under the directory corresponding to the device class (in this case,
/sys/class/arcom-examples/).

#include <linux/init.h> #define DEVICE_NAME "arcom-chrdev"

#include <linux/module.h> #define DEVICE_FILE_NAME "arcom-chrdev"
#include <linux/kernel.h> #define CLASS "arcom-examples"

MODULE_AUTHOR("ARCOM 2018/19"); static dev_t dev_num;

static struct class *arcom_class;
MODULE_LICENSE("GPL");
static struct cdev arcom_cdev[NDEVICES];

static int __init arcom_chrdev_init(void) {

pr_info("Module loaded\n"); struct file_operations arcom_fops = {
return 0; open: arcom_chrdev_open,
} release: arcom_chrdev_release,
read: arcom_chrdev_read,
write: arcom_chrdev_write,
static void __exit arcom_chrdev_exit(void) {
};
pr_info("Module unloaded\n");
} static int __init arcom_chrdev_init(void)
{
module_init(arcom_chrdev_init); // int init_module(void) int i;
module_exit(arcom_chrdev_exit); // void cleanup_module(void) dev_t curr_dev;

/* Request the kernel for NDEVICES minor numbers*/

alloc_chrdev_region(&dev_num, 0, NDEVICES, DEVICE_NAME);

/* Create a new class for this device. It will be visible in /sys/class */

arcom_class = class_create(THIS_MODULE, CLASS);

/* Create NDEVICES character devices (cdev)

* Each device will be accessible via /dev/arcom-chrdevX, where X
* is the respective minor number
*/
for (i = 0; i < NDEVICES; i++) {

/* Tie file_operations to the cdev */

cdev_init(&arcom_cdev[i], &arcom_fops);
arcom_cdev[i].owner = THIS_MODULE;

1
sysfs is a non-persistent virtual filesystem, usually mounted in /sys, that provides a global view of the system
and exposes the kernel object's attributes and overall topology.

Loadable Kernel Module 1/12 Loadable Kernel Module 2/12

ARCOM – MEEC – ISEP – 2020/2021 ARCOM – MEEC – ISEP – 2020/2021
 2) Build the module for your Linux distribution, using the typical Makefile:
/* Device number to be associated to the current device */
curr_dev = MKDEV(MAJOR(dev_num), MINOR(dev_num) + i); obj-m +=chrdev.o
LKM_DIR=/lib/modules/$(shell uname -r)/build
/* Activate the device */
cdev_add(&arcom_cdev[i], curr_dev, 1); all:
make -C ${LKM_DIR} M=$(PWD) modules
/* Device info can be viewed under /sys/class/arcom-example */ clean:
device_create(arcom_class, make -C ${LKM_DIR} M=$(PWD) clean
NULL, /* no parent device */
curr_dev,
Note that your distribution should have the matching kernel, kernel-devel and kernel-
NULL, /* no additional data */
DEVICE_FILE_NAME "%d", i); headers packages:
}
dnf install kernel-headers kernel-devel
pr_info("Module loaded\n"); dnf update kernel kernel-headers kernel-devel
return 0;
} If the last command leads to a kernel update, you should reboot the system to be able to compile
and load the modules.
static void __exit arcom_chrdev_exit(void)
3) Open another console to work as root.
{
int i;
dev_t curr_dev; 3.1) Start dmesg in background so that you can see the kernel messages:

unregister_chrdev_region(MKDEV(MAJOR(dev_num), MINOR(dev_num)), NDEVICES); dmesg -wH &

for (i = 0; i < NDEVICES; i++) { 3.2) Load the module:

curr_dev = MKDEV(MAJOR(dev_num), MINOR(dev_num) + i);
device_destroy(arcom_class, curr_dev);
insmod chrdev.ko
cdev_del(arcom_cdev + i);
}
If this command fails due to “kernel lockdown”2, run mokutil --disable-validation
class_destroy(arcom_class); (define an easy to remember password) and reboot the system.
pr_info("Module unloaded\n"); 4) Check the contents of /sys/class/arcom-examples/ and find out which major number
}
and minor numbers were assigned to each character device (this information is stored in the
1) The above code is provided at the website, as chrdev.c. Complete the source code with uevent file located in the directory associated to each device).
the definition of the file operations functions. At this stage, each function should simply print
a message. As an example, arcom_chrdev_open is already defined as follows: 5) Check if the device files were created in /dev:

int arcom_chrdev_open(struct inode * inode, struct file * filp) ls -l /dev | grep arcom-chrdev
{
pr_info("Device open\n"); 6) Perform a read operation on the device file:
return 0;
} cat /dev/arcom-chrdev0

2
See “man kernel_lockdown” and https://fedoraproject.org/wiki/Secureboot

Loadable Kernel Module 3/12 Loadable Kernel Module 4/12

ARCOM – MEEC – ISEP – 2020/2021 ARCOM – MEEC – ISEP – 2020/2021
Input/output between the module and user space processes
at a time. To achieve that behavior, use the wait_queue mechanism:
In what follows, assume that NDEVICES equates to 1.
7) Next, we will change the file operation functions so that each device file behaves as a data
file limited to a maximum size of 256 bytes:
• Data sent to the device should be saved in an internal buffer.
• Read operations should return the content of the buffer
• Data should be kept in the buffer after the device is closed, to be accessible next time
the device is open.
7.1) Consider the following definition for the write function:
#define MAX_FILE_SIZE 256
static int size; //initialize to 0 in module_init
static char data[MAX_FILE_SIZE];
8.1) Change the open and close functions so that only one process may access the device file
/* Declaration of the wait queue variable (wait_queue_head_t data type)
* This must be added to your chrdev.c file
*/
DECLARE_WAIT_QUEUE_HEAD(name);
The following function prototypes are presented here for reference only. Do not
copy them to the chrdev.c file.
// Initialization of previously declared wait_queue_head_t variable
void init_waitqueue_head(wait_queue_head_t *queue); //
// Block the current task in the wait queue if CONDITION is false
int wait_event_interruptible(wait_queue_head_t q, CONDITION);
//Make the other tasks check if CONDITION above has become true
void wake_up_interruptible(wait_queue_head_t *q);

ssize_t arcom_chrdev_write(struct file * filp, const char __user * buf, The condition for wait_event_interruptible may simply be the test of a flag variable,
size_t count, loff_t *offset)
for instance:
{
int i, n;
int deviceOpen = 0;
int last = *offset + count;

if(last > MAX_FILE_SIZE) This variable should be set to 1 when the access to the device is granted to a process, and to 0
last = MAX_FILE_SIZE; when the process closes the device. Then, upon each call to the open function, the module
should check the condition like this:
i = last - *offset;
n = copy_from_user(data + *offset, buf, i); wait_event_interruptible(myWaitQueue, !deviceOpen);
if(n!=0) {
pr_warning("Could not copy %d bytes\n", n); Moreover, the module should call wake_up_interruptible before returning from the
return -EFAULT;
}
release function.
*offset += i;
8.2) Test the new feature using the following program (open_dev.c, provided at the website):
if(size < *offset) //keep data from previous accesses
size = *offset; void handler(int s) {
printf("Received signal %d\n", s);
return i; };
}
int main(int argc, char *argv[]) {
int fd;
7.2) Implement the arcom_chrdev_read function.
signal(SIGINT, handler); //handler for SIGINT (signal generated by CTRL+C)
printf("Process %d\n", getpid());
Module locking/unlocking if((argc > 1) && strcmp(argv[1], "-NONBLOCK")==0)
fd = open("/dev/arcom-chrdev0", O_RDWR | O_NONBLOCK);
7.3) The functions try_module_get and module_put are used to update an internal counter else
of processes using the module. This is useful to ensure the module is not unloaded while a fd = open("/dev/arcom-chrdev0", O_RDWR);
device is in use. To achieve that goal, call try_module_get(THIS_MODULE) at the if(fd<0) {
perror("open");
beginning of the open function, and module_put(THIS_MODULE) before returning from the return 1;
release function. }
printf("File descriptor: %d\n", fd);
pause(); //wait until a signal is received
return 0;
}

Loadable Kernel Module 5/12 Loadable Kernel Module 6/12

ARCOM – MEEC – ISEP – 2020/2021 ARCOM – MEEC – ISEP – 2020/2021
8.3.1) Build the program and run it in both terminals. The second time the program is executed, 9.1) Change the module code so that, for the case of NDEVICES greater than 1, each device has
it should block until you terminate the process launched in the first place. Terminate the first its own data buffer. In order to achieve that, the module must identify which device is being
process by pressing CTRL+C in the respective terminal. accessed by the file operation. In the open and release operations this can be done as
follows:
8.3.2) Execute the program again and press CTRL+C. This time, the process will open the
device file in spite the fact it is already open by another process. This is not the desired behavior, typedef struct
{
and this happens because wait_event_interruptible returns whenever the calling task is
struct cdev arcom_cdev;
interrupted by a signal. In these cases, it is not desirable to mark the device as open, otherwise char data[MAX_FILE_SIZE];
the device will be inaccessible to other tasks until the process terminates or until it closes the int size;
file. Instead, the open operation should unlock the module and return -EINTR: int deviceOpen;
wait_queue_head_t WaitQ; //initialize with init_waitqueue_head
wait_event_interruptible(myWaitQueue, !deviceOpen); } arcom_chrdev_t;
if (deviceOpen) { // If device is still open, that means
// wait_event_interruptible was interrupted by a signal static arcom_chrdev_t devices[NDEVICES];
module_put(THIS_MODULE); //unlock the module
return -EINTR; int arcom_chrdev_open(struct inode * inode, struct file * filp)
} {
unsigned int maj = imajor(inode);
Add the above change to the module and repeat the previous test. Do not forget to unload unsigned int min = iminor(inode);
(rmmod), build (make) and load (insmod) the module again.
arcom_chrdev_t *ptr;
ptr = container_of(inode->i_cdev, arcom_chrdev_t, arcom_cdev);
8.4) The open API provides the O_NONBLOCK flag to request a nonblocking behavior. Change filp->private_data = ptr;
the module so that the O_NONBLOCK flag is honored:
In the module_init function, the cdev data for each device should be stored in the
if ((filp->f_flags & O_NONBLOCK) && deviceOpen) arcom_cdev field of the corresponding element in the devices vector. Given a pointer to the
{
pr_info("Would block\n");
arcom_cdev field (which is provided in the i_cdev field of the inode parameter of the open
module_put(THIS_MODULE); //unlock the module and release operations), the container_of function returns a pointer to the variable it
return -EAGAIN; belongs to, which in this case is the corresponding element in the devices vector. This pointer
} is stored in the private_data field of the the filp parameter, which is passed to the read
and write operations:
After rebuilding and reloading the module, execute the test program in one first terminal as
before and, in the other terminal, execute the test program with the -NONBLOCK command line ssize_t arcom_chrdev_read (struct file *filp, char __user * buf, size_t count,
argument: loff_t * offset)
{
open_dev -NONBLOCK arcom_chrdev_t *ptr = filp->private_data;

9.2) Change NDEVICES to 2 and test the module as described next. In the first terminal:

./open_dev

In the second terminal:

cat /dev/arcom-chrdev0 //should block, press CTRL+C
echo test1 > /dev/arcom-chrdev1
cat /dev/arcom-chrdev1

In the first terminal, press CTRL+C to terminate open_dev and then:

echo test0 > /dev/arcom-chrdev0

cat /dev/arcom-chrdev0
cat /dev/arcom-chrdev1

Loadable Kernel Module 7/12 Loadable Kernel Module 8/12

ARCOM – MEEC – ISEP – 2020/2021 ARCOM – MEEC – ISEP – 2020/2021
Device file permissions Appendix A – API Documentation

10) Execute the following command as a standard user: int alloc_chrdev_region(dev_t * dev, unsigned baseminor, unsigned count, const char
* name)
cat /dev/arcom-chrdev0
register a range of char device numbers
10.1) The command will fail due to lack of permissions. It is possible to change the permissions
manually using chmod, however, if the module must be accessed frequently by non-privileged
Parameters
users, that would quickly become a tedious task and would eventually end up requiring adding
a new command to the system startup scripts. dev_t * dev
For distributions based on udev, it is possible to automatically change the permissions of a output parameter for first assigned number
(automatically created) device file by creating a file with the corresponding rule in unsigned baseminor
/etc/udev/rules.d: first of the requested range of minor numbers
unsigned count
# nano /etc/udev/rules.d/70-arcom.rules the number of minor numbers required
KERNEL=="arcom-chrdev*", MODE="0666" const char * name
the name of the associated device or driver
In this case, the rule gives read and write permissions for all users for device files whose name
has the arcom-chrdev prefix.
Description
After creating the file listed above, run udevadm control --reload-rules (to make sure
Allocates a range of char device numbers. The major number will be chosen dynamically,
udev loads the new rule) and reload the module: and returned (along with the first minor number) in dev. Returns zero or a negative error
udevadm control --reload-rules #required only after changing the rules code.
rmmod chrdev
insmod chrdev.ko
struct device * device_create(struct class * class, struct device * parent, dev_t devt, void
The device file permissions should now be crw-rw-rw-. In the standard user console: * drvdata, const char * fmt, ...)
ls -l /dev | grep arcom-chrdev
cat /dev/arcom-chrdev0
creates a device and registers it with sysfs

10.2) In general, file permissions should be handled by udev. However, as an alternative, the Parameters
default permissions can be set by the module itself (note, however, that these settings are
struct class * class
overridden by udev in case any rule, as the one defined above, is applicable):
pointer to the struct class that this device should be registered to
struct device * parent
static char *arcom_devnode(struct device *dev, umode_t *mode)
{ pointer to the parent struct device of this new device, if any
if (!mode) dev_t devt
return NULL; the dev_t for the char device to be added
void * drvdata
if (MAJOR(dev->devt) == MAJOR(dev_num)) the data to be added to the device for callbacks
*mode = 0666; const char * fmt
string for the device’s name
return NULL; ...
} variable arguments
static int __init arcom_chrdev_init(void)
{ Description

(…) This function can be used by char device classes. A struct device will be created in sysfs,
registered to the specified class.
arcom_class = class_create(THIS_MODULE, CLASS);
arcom_class->devnode = arcom_devnode;
A “dev” file will be created, showing the dev_t for the device, if the dev_t is not 0,0. If a
pointer to a parent struct device is passed in, the newly created struct device will be a child
Loadable Kernel Module 9/12 Loadable Kernel Module 10/12
ARCOM – MEEC – ISEP – 2020/2021 ARCOM – MEEC – ISEP – 2020/2021
of that device in sysfs. The pointer to the struct device will be returned from the call. Any
further sysfs files that might be required can be created using this pointer. Bibliography
Returns struct device pointer on success, or ERR_PTR() on error. • Linux Device Drivers Development, published by Packt, 2017,
https://github.com/PacktPublishing/Linux-Device-Drivers-
Note Development/blob/master/Chapter04/dummy-char.c
• Jonathan Corbet, Alessandro Rubini, and Greg Kroah-Hartman, Linux Device
the struct class passed to this function must have previously been created with a call to Drivers, Third Edition, 2005, https://lwn.net/Kernel/LDD3/
class_create(). • The Linux Documentation, https://www.kernel.org/doc/html/latest/index.html
• Peter Jay Salzman, Michael Burian, Ori Pomerantz, The Linux Kernel Module
Programming Guide, 2001
container_of(ptr, type, member)

cast a member of a structure out to the containing structure Document history

• 2018-11-19 – Created by Jorge Estrela da Silva (jes@isep.ipp.pt)
Parameters • 2019-09-02 – Moved udev section to the end of the document. Jorge Estrela da Silva
(jes@isep.ipp.pt)
ptr
• 2020-12-24 – Added installation/update of kernel development modules, reference to
the pointer to the member. mokutil command, extended explanation for application of the wait queue. Jorge
type
Estrela da Silva (jes@isep.ipp.pt)
the type of the container struct this is embedded in.
member
the name of the member within the struct.

void init_waitqueue_head(wait_queue_head_t *queue);

DECLARE_WAIT_QUEUE_HEAD(queue);

The defined type for Linux wait queues. A wait_queue_head_t must be

explicitly initialized with either init_waitqueue_head at runtime or
DECLARE_WAIT_QUEUE_HEAD at compile time.

Loadable Kernel Module 11/12 Loadable Kernel Module 12/12

ARCOM – MEEC – ISEP – 2020/2021 ARCOM – MEEC – ISEP – 2020/2021