[Howto] Multiclick button handler for 3.5" LCD and webcam

Moderators: odroid, mdrjr

[Howto] Multiclick button handler for 3.5" LCD and webcam

Unread postby mad_ady » Fri Mar 31, 2017 9:55 pm

One thing I really liked about the Odroid 3.5" touchscreen module is the inclusion of four hardware switches that could be programmed to do just about anything. But there is one thing I dislike - there are only 4 buttons available. Can we do anything about that?

In order to get started using the 3.5" touchscreen, I found it best to follow @Fourdee's guide at viewtopic.php?f=145&t=24248. This will set up the touchscreen, but doesn't use the buttons for anything. HardKernel provide on their wiki sample code to convert the 4 buttons in a virtual keyboard that can output 4 key presses: http://odroid.com/dokuwiki/doku.php?id= ... lcd_shield, but we won't need it for our purposes.

My first idea of getting more out of these buttons was to see if I could click two buttons simultaneously and generate a different event. The way the buttons work is to create a "short" between the ADC pin and ground through one or more resistors. Based on the resulting resistance value the ADC produces a numerical value which helps you identify which button has been pressed.

Image
Figure 1 (a) The original HK design
Image
Figure 1 (b) improvement option

Figure 1a shows the design HardKernel chose while Figure 1b shows a small modification which could have added 3 more "events" from these buttons. Had the resistors been mounted closer to the ground (instead of before the switch) you could have created a parallel circuit when pressing two buttons and have a different ADC value. I did the math (I was surprised I still knew how 10 years after college) and worked out that for a C1/C2 the following values would have been possible with the reversed resistors:

Image
Figure 2 - ADC values for multiplexed buttons

One possible issue with this approach might have been getting "misclicks" if the buttons were not pressed simultaneously, but since HardKernel didn't implement the hardware this way, we need to explore other alternatives.

A different way to handle key events is to handle multiple key presses as a single event. For example, you could handle a "double/triple-click" event or even a "long-press" and have it execute something else. The way you can detect such an event is to set up a buffer, record key presses in the buffer and when the buffer is full process the event.

Since this seemed like a fun and simple project my plan was to expand HardKernel's C code that handled the keys and add this functionality - however, it seems I am too old to go back to C. I've been spoiled by too many years in higher-level programming languages that handle a lot of complexity behind the programmer's back to manage to do it in C. So I did it in perl…

The code works a bit like this - there's an infinite loop reading the ADC input. Once a change has been detected (a key has been pressed), the value is recorded in a buffer. The following reads are also stored in the buffer (even when no key is pressed) and when the buffer is full the key sequence is identified and some action is taken. This approach had the fortunate side-effect that it adds support for key sequences such as "KEY1-KEY2" or "KEY1-KEY2-KEY3".

To get the code and set it up follow the steps below:
Code: Select all
git clone https://github.com/mad-ady/tftlcd35-key.git
cd tftlcd35-key
sudo apt-get install liblog-log4perl-perl libproc-background-perl libconfig-yaml-perl
sudo cp tftlcd35_key.pl /usr/local/bin
sudo cp tftlcd35-key.service /etc/systemd/system/


The code ships with a few configuration examples to use based on your needs. If you only want one click and one long-click per key use config-empty-1.yaml, if you want to use two key combinations use config-empty-2.yaml or if you want 3 key combinations use config-empty-3.yaml. The configuration syntax is YAML which is a simple format, but like python indentation counts (so no TABs please). If starting the script generates complaints about the configuration you can use an online validator like http://www.yamllint.com/ to pinpoint the problem. The default config sets logging to INFO (if you have issues you can set it to DEBUG, but it's very verbose), sets the period between reads to 200ms, the buffer length to 10 and the longpress interval to 70% of the buffer length.

To understand how these parameters affect you let's analyze the following hypothetical buffer:
Image
Figure 3 - Sample buffer

Each cell represents a read value from the ADC. The buffer has 10 cells, so bufferSize has to be 10. Depending on how fast you press and release a key you can set the updatePeriod higher or lower (default is 200000 microseconds, or 200ms). If you set updatePeriod to something too long (e.g. 500ms) you will miss keypresses because you can press and release the button when the script is sleeping. If you set updatePeriod to something too short you may need a bigger buffer to record all your key presses. For instance, if you get the buffer above, but you double-clicked KEY1, this means your updatePeriod is too high.

From the second you press a key the script will take updatePeriod * bufferSize microseconds to react. This means that having a high bufferSize gives you a high reaction time (you pressed the button once, but the action happens 5 seconds later). The default configuration uses a 2 second reaction time (200ms * 10).

The final global parameter is longPress which represents how many items in the buffer have to be a certain key before the buffer is interpreted like a long press. The default is 0.7 - which means 70% of the buffer has to be filled with a key press (for our example it would mean holding down a key for at least 1.4s). Note that if you hold down the key for too long (e.g. 2.2s) and the buffer size is exceeded, the program will interpret the first 2s as a long press and generate the desired event and the final 0.2s as a short key press and fire a different event! This can be mitigated in the code with a short sleep after a long event if needed.

The final part of the configuration is the key to command mapping. Key sequences are separated by a dash (-), and long presses are prefixed by LONG. In the example above the key sequence interpreted by the script is KEY1-KEY2-KEY3 (duplicated keys are ignored). To get a KEY1-KEY1-KEY1 sequence you need to release KEY1 for at least an updatePeriod so that the buffer contains a blank reading between two keys.

To map a command to a sequence, simply type in the command you want executed on the same line after the ':' sign. The commands are executed in a background shell as the same user as the script runs (root). If you need to run a graphical command you can prefix it with DISPLAY=:0.

Image
Figure 4: Example configuration

Once the configuration is finished you can copy it as /etc/tftlcd35-key.yaml, activate the script and have it start up automatically:
Code: Select all
sudo cp custom-config.yaml /etc/tftlcd35-key.yaml
sudo systemctl enable tftlcd35-key
sudo systemctl start tftlcd35-key

Logging and debugging information goes to syslog and can be viewed with
Code: Select all
sudo journalctl -f -u tftlcd35-key


Using a button on a device
If you don't have HardKernel's 3.5" display to play with, but you have devices with physical buttons around (like a camera or a sound card) you can still get multiple actions from them in a similar way. For example, the HardKernel 720p camera comes with a button to take snapshots, which is rarely used. The button registers as an input device (e.g. standard keyboard) and registers only one button as an event. You can easily find out which keys are supported by running the evtest command as shown in figure 5.
Code: Select all
sudo apt-get install evtest


Image
Figure 5: evtest sample output

The input devices are mapped in /dev/input/event*, but the mapping is dynamic so you can't depend on the numbers across reboots. Instead you should identify your device from /dev/input/by-id/ which is more stable (figure 6).

Image
Figure 6: Stable input mapping

So, to benefit from click and multi-click events (double/triple/long-press) you can download and install this handler program:
Code: Select all
git clone https://github.com/mad-ady/multibutton.git
cd multibutton
sudo perl -MCPAN -e 'install Linux::Input'
sudo apt-get install libconfig-yaml-perl liblog-log4perl-perl libproc-background-perl
sudo cp multibutton.pl /usr/local/bin
sudo cp config-minimal.yaml /etc/multibutton.yaml
sudo cp multibutton.service /etc/systemd/system


Before starting the service you will need to configure it. First step is to identify your input device and provide the correct path to it in /etc/systemd/system/multibutton.service (the -i parameter), as described above.
Next, tweak the configuration file to your needs. The syntax is the same as the touchscreen handler, but since in my case I only have one button, there is only one thing I can control. The configuration file is in yaml format and has the same 4 configuration options which tweak polling period, buffer size and long press percentage. In this case updatePeriod is not as important, since the kernel will do the polling and will forward all events to the end user (no more missed keys). Next comes the key sequences in the format "KEY_LABEL1-KEY_LABEL2". You can get the correct labels with the evtest command. Following the key labels you need to specify the command(s) to be run. Below is an example (see the actual file for comments):

Image
Figure 7: multibutton.pl sample configuration

Once you've set up your configuration and systemd service you can turn the service on with:
Code: Select all
sudo systemctl daemon-reload
sudo systemctl enable multibutton
sudo systemctl start multibutton

You can view the logs with journalctl:
Code: Select all
sudo journalctl -u multibutton -f

If you need to support multiple devices (e.g. two cameras) you can create a new service, point it to the new device and load a different configuration.

Note that you can run this event handler on any input device - including a real keyboard and only configure certain key presses to execute various commands. This could make for an interesting pranking alternative where for example you bind "w" to execute some arbitrary command when pressed three times in a row. But note that the code is untested for handling multiple keys :)

Happy clicking!
User avatar
mad_ady
 
Posts: 4737
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2, N1

Re: [Howto] Multiclick button handler for 3.5" LCD and webca

Unread postby odroid » Sat Apr 01, 2017 8:41 am

Very nice and useful How-to indeed.
We should choose the second schematics for easier/simpler multiclicks. :oops:

Anyway, Thank you for sharing a nice guide.
I've added a link into our WiKi. :D
http://odroid.com/dokuwiki/doku.php?id= ... plications
User avatar
odroid
Site Admin
 
Posts: 28890
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: [Howto] Multiclick button handler for 3.5" LCD and webca

Unread postby mad_ady » Sat Apr 01, 2017 1:48 pm

Not sure if the second schematics has any drawbacks. Anyway, clicking two buttons simultaneously is risky. You might read KEY1|KEY1+KEY2|KEY2 depending on when/how you press/release them. Also having 3 key combinations is pretty overkill - you get 88 total key combinations - enough to map an entire keyboard :)
User avatar
mad_ady
 
Posts: 4737
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2, N1

Re: [Howto] Multiclick button handler for 3.5" LCD and webca

Unread postby mad_ady » Mon Apr 03, 2017 11:33 pm

I found a bug in the multibutton script. It relies on the perl module Linux::Input to get inputs, but if you reset the USB bus or if you unplug the USB input device, the module hangs (and hogs a CPU to 100% usage) without any way to detect this from inside the application (I tried timing the polling loops to see if they end faster and also I tried checking the inode of the /dev/input device). The behavior is that once the device is gone, the script "freezes" and any checks I attempt inside no longer run.
So, I implemented a crude workaround using systemd. I set systemd WatchdogSec value to 300 (5 minutes), so the script is restarted every 5 minutes. This should limit the "damage" that can be caused by unplugging a device. Will need to check what it does for long-running processes (but so far I'm only starting short scripts).
The changes are already on github.

Please let me know if you run into other issues
User avatar
mad_ady
 
Posts: 4737
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2, N1

Re: [Howto] Multiclick button handler for 3.5" LCD and webca

Unread postby mad_ady » Fri Apr 07, 2017 9:12 pm

Hmm, I've noticed something strange. From time to time I get a "KEY1 was pressed" event on a C2 even if the LCD screen is not yet installed:
Code: Select all
Apr 06 21:56:32 uy-scuti tftlcd35_key.pl[670]: 2017/04/06 21:56:32 Processing buffer: KEY1| | | | | | | | | | | | | | |
Apr 06 23:49:15 uy-scuti tftlcd35_key.pl[670]: 2017/04/06 23:49:15 Processing buffer: KEY1| | | | | | | | | | | | | | |
Apr 07 04:32:14 uy-scuti tftlcd35_key.pl[670]: 2017/04/07 04:32:14 Processing buffer: KEY1| | | | | | | | | | | | | | |
Apr 07 14:53:01 uy-scuti tftlcd35_key.pl[670]: 2017/04/07 14:53:01 Processing buffer: KEY1| | | | | | | | | | | | | | |



The strange thing is that the value for KEY1 is around 5, while the value for the "nothing pressed" is around 1024:
Code: Select all
2017/04/07 15:09:52 Read 1020
2017/04/07 15:09:52 Read 976
2017/04/07 15:09:52 Read 945
2017/04/07 15:09:52 Read 928
2017/04/07 15:09:52 Read 942
2017/04/07 15:09:52 Read 985
2017/04/07 15:09:52 Read 1013
2017/04/07 15:09:52 Read 1022
2017/04/07 15:09:52 Read 1020
2017/04/07 15:09:53 Read 1022
2017/04/07 15:09:53 Read 1021
2017/04/07 15:09:53 Read 1023
2017/04/07 15:09:53 Read 1023
2017/04/07 15:09:53 Read 1023
2017/04/07 15:09:53 Read 1023
2017/04/07 15:09:53 Read 1020
2017/04/07 15:09:53 Read 1020
2017/04/07 15:09:53 Read 985
2017/04/07 15:09:53 Read 927
2017/04/07 15:09:54 Read 1023
2017/04/07 15:09:54 Read 1023
2017/04/07 15:09:54 Read 1023
2017/04/07 15:09:54 Read 1023
2017/04/07 15:09:54 Read 1022
2017/04/07 15:09:54 Read 1020
2017/04/07 15:09:54 Read 1018
2017/04/07 15:09:54 Read 950
2017/04/07 15:09:54 Read 956
2017/04/07 15:09:55 Read 1021
2017/04/07 15:09:55 Read 1022


Maybe the problem is that the ADC pin is floating and thus may have a variable potential. Once it is connected to the display, it should be stabilized. We'll see if the problem reappears once I connect the display.
User avatar
mad_ady
 
Posts: 4737
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2, N1

Re: [Howto] Multiclick button handler for 3.5" LCD and webca

Unread postby odroid » Sat Apr 08, 2017 11:03 am

Yes. The input impedance is quite high(>5Momhs) on ADC pin and it must cause the problem.
If you touch your finger on the pin, the ADC readout varies a lot.
User avatar
odroid
Site Admin
 
Posts: 28890
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: [Howto] Multiclick button handler for 3.5" LCD and webca

Unread postby mad_ady » Wed Apr 19, 2017 9:10 pm

Hmm, the problem reappeared - the screen is connected to the C2 and rarely (once every 1-2 days) I get the KEY1 event detected:
Code: Select all
Apr 19 07:05:31 uy-scuti tftlcd35_key.pl[585]: 2017/04/19 07:05:31 KEY1 is pressed (raw value 0
Apr 19 07:05:31 uy-scuti tftlcd35_key.pl[585]: )
Apr 19 07:05:33 uy-scuti tftlcd35_key.pl[585]: 2017/04/19 07:05:33 Processing buffer: KEY1| | | | | | | | | | | | | | |
Apr 19 07:05:33 uy-scuti tftlcd35_key.pl[585]: 2017/04/19 07:05:33 Identified key sequence: KEY1

At the time of the log I wasn't near the keypad, but I heard MPD starting to play. Not sure why I read 0 from the ADC...
I need to figure out a way to filter out such events. Maybe force KEY1 to appear at least twice in a sequence, or ignore KEY1 without it being followed by anything else (a short KEY1 event)... This happens only for KEY1, so it must be something electrical which causes the ADC to read 0...
User avatar
mad_ady
 
Posts: 4737
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2, N1


Return to 3.5inch LCD Shield

Who is online

Users browsing this forum: No registered users and 1 guest