C2 watchdog

Post Reply
Bl4ckD0g
Posts: 48
Joined: Sat Apr 09, 2016 3:18 am
languages_spoken: english
ODROIDs: C2
Contact:

C2 watchdog

Unread post by Bl4ckD0g » Sat Apr 09, 2016 3:22 am

Hi,

Does this board/chip support watchdog?
If so, do you plan to implement/support it in the kernel?

It would be very helpful to have it enabled as it could restart the system in case of a hang.

Thanks in advance.

Bl4ckD0g
Posts: 48
Joined: Sat Apr 09, 2016 3:18 am
languages_spoken: english
ODROIDs: C2
Contact:

Re: C2 watchdog

Unread post by Bl4ckD0g » Tue Apr 12, 2016 10:46 am

Anyone?

I understand that C1 has the watchdog enabled in the kernel.
Is there any plan to implement the same in the C2?

If you need any further detail on my questions please let me know.

Thanks!

User avatar
odroid
Site Admin
Posts: 30050
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Contact:

Re: C2 watchdog

Unread post by odroid » Tue Apr 12, 2016 6:14 pm

We will look into the feasibility of WDT in May.

Bl4ckD0g
Posts: 48
Joined: Sat Apr 09, 2016 3:18 am
languages_spoken: english
ODROIDs: C2
Contact:

Re: C2 watchdog

Unread post by Bl4ckD0g » Tue Apr 12, 2016 8:03 pm

Sounds great!

Only to confirm, does it have the hw capabilities and is it a matter of implementing the driver?

Thanks :)

User avatar
odroid
Site Admin
Posts: 30050
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Contact:

Re: C2 watchdog

Unread post by odroid » Wed Apr 13, 2016 10:15 am

We are checking the availability of the WDT block in the S905 SoC now.
I will share the information within a couple of days to know the feasibility together.

Bl4ckD0g
Posts: 48
Joined: Sat Apr 09, 2016 3:18 am
languages_spoken: english
ODROIDs: C2
Contact:

Re: C2 watchdog

Unread post by Bl4ckD0g » Wed Apr 13, 2016 8:20 pm

Ok! Looking forward to it.
Sounds promising.

Thanks!

Paint
Posts: 88
Joined: Tue Mar 15, 2016 7:41 am
languages_spoken: English
ODROIDs: C2 with 64gb eMMC
Contact:

Re: C2 watchdog

Unread post by Paint » Tue Apr 19, 2016 11:07 pm

odroid wrote:We are checking the availability of the WDT block in the S905 SoC now.
I will share the information within a couple of days to know the feasibility together.
Great! I look forward to this. I already enabled the watchdog functionality within my kernel fork but havent setup any watchdog process:

https://github.com/JoshDi/linux/tree/odroidc2-3.14.y

User avatar
odroid
Site Admin
Posts: 30050
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Contact:

Re: C2 watchdog

Unread post by odroid » Wed Apr 20, 2016 11:01 am

Yes, we can say there is a WDT function in the S905 SoC at least. :)

Paint
Posts: 88
Joined: Tue Mar 15, 2016 7:41 am
languages_spoken: English
ODROIDs: C2 with 64gb eMMC
Contact:

Re: C2 watchdog

Unread post by Paint » Wed Apr 20, 2016 1:53 pm

odroid wrote:Yes, we can say there is a WDT function in the S905 SoC at least. :)
Odroid, do we just need to modify the dts file?

http://forum.odroid.com/viewtopic.php?f=115&t=8984

User avatar
odroid
Site Admin
Posts: 30050
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Contact:

Re: C2 watchdog

Unread post by odroid » Wed Apr 20, 2016 2:27 pm

Probably yes.
Please try it and let us know the result if you can.

We have some urgent items to implement/fix by early May.

Bl4ckD0g
Posts: 48
Joined: Sat Apr 09, 2016 3:18 am
languages_spoken: english
ODROIDs: C2
Contact:

Re: C2 watchdog

Unread post by Bl4ckD0g » Fri Apr 29, 2016 8:36 pm

Great!
Looking forward to it.

I am using official kernel, so waiting for it to be released in the official channel.

nh1402
Posts: 11
Joined: Wed May 18, 2016 5:21 pm
languages_spoken: english
ODROIDs: Odroid C2
Contact:

Re: C2 watchdog

Unread post by nh1402 » Fri May 20, 2016 12:18 am

Any update on this?

Bl4ckD0g
Posts: 48
Joined: Sat Apr 09, 2016 3:18 am
languages_spoken: english
ODROIDs: C2
Contact:

Re: C2 watchdog

Unread post by Bl4ckD0g » Mon May 30, 2016 9:09 pm

Is there any plan to support the watchdog in the kernel?

It would help a lot to automatically reboot the C2 when we have hung states like the issue with the setting "nographics" set to 1:

http://forum.odroid.com/viewtopic.php?f=136&t=21520

nh1402
Posts: 11
Joined: Wed May 18, 2016 5:21 pm
languages_spoken: english
ODROIDs: Odroid C2
Contact:

Re: C2 watchdog

Unread post by nh1402 » Tue Jun 07, 2016 7:43 pm

This is now my main stumbling block from using the C2.

Pepes
Posts: 98
Joined: Mon Feb 15, 2016 5:28 am
languages_spoken: Czech | English
ODROIDs: C2 and U2
-->both without eMMC (overpriced)<--
Location: Praha | Czech republic
Contact:

Re: C2 watchdog

Unread post by Pepes » Sat Jun 11, 2016 6:24 pm

I saw watchdog enabled in mdrjr's repo

nh1402
Posts: 11
Joined: Wed May 18, 2016 5:21 pm
languages_spoken: english
ODROIDs: Odroid C2
Contact:

Re: C2 watchdog

Unread post by nh1402 » Mon Jun 13, 2016 4:49 pm

That's the main repo actually, so we'll probably see it in the next kernel update.

moon.linux
Posts: 1162
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english
Contact:

Re: C2 watchdog

Unread post by moon.linux » Thu Jun 23, 2016 1:06 pm

Yes watchdog driver need more update to work correctly, but when running watchdog demon it dose not trigger automatic reboot.

Bl4ckD0g
Posts: 48
Joined: Sat Apr 09, 2016 3:18 am
languages_spoken: english
ODROIDs: C2
Contact:

Re: C2 watchdog

Unread post by Bl4ckD0g » Fri Jun 24, 2016 3:41 am

Was it added into any recent kernel release?

Would it use the same procedure as the one for the C1 to enable it?
http://odroid.com/dokuwiki/doku.php?id= ... hdog_timer

moon.linux
Posts: 1162
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english
Contact:

Re: C2 watchdog

Unread post by moon.linux » Fri Jun 24, 2016 2:27 pm

Actually I am working on the the same way above to setup watchdog process to keep alive the device.
But when we try some thing like this without watchdog demon not configures.

Code: Select all

root@odroid64:~# echo 6 > /dev/watchdog0
[   62.271068] watchdog watchdog0: watchdog did not stop!
root@odroid64:~#
root@odroid64:~#
root@odroid64:~#
root@odroid64:~#
root@odroid64:~# GXBB:BL1:08dafd:0a8993;FEAT:EDFC318C;POC:3;RCY:0;EMMC:800;NAND:81;SD:0;READ:0;CHK:0;
TE: 112493
no sdio debug board detected

BL2 Built : 11:44:26, Nov 25 2015.
gxb gfb13a3b-c2 - jcao@wonton

Board ID = 8
set vcck to 1100 mv
set vddee to 1050 mv
CPU clk: 1536MHz
DDR channel setting: DDR0 Rank0+1 same
DDR0: 2048MB(auto) @ 912MHz(2T)-13
DataBus test pass!
AddrBus test pass!
Load fip header from SD, src: 0x0000c200, des: 0x01400000, size: 0x000000b0
Load bl30 from SD, src: 0x00010200, des: 0x01000000, size: 0x00009ef0
Sending bl30........................................OK.
Run bl30...
Load bl301 from SD, src: 0x0001c200, des: 0x01000000, size: 0x000017c0
Wait bl30...Done
Sending bl301......OK.
Run bl301...
31 from SD, src: 0x00020200, des: 0x10100000, size: 0x00011130
When configured watchdog demon and we try to kill the watchdog process it dose not trigger reset and reboot.

Code: Select all

odroid@odroid64:~$ service watchdog status
● watchdog.service - watchdog daemon
   Loaded: loaded (/lib/systemd/system/watchdog.service; static; vendor preset:
   Active: active (running) since Fri 2016-06-24 01:23:41 EDT; 3s ago
  Process: 1403 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/wa
  Process: 1400 ExecStartPre=/bin/sh -c [ -z "${watchdog_module}" ] || [ "${watc
 Main PID: 1405 (watchdog)
   CGroup: /system.slice/watchdog.service
           └─1405 /usr/sbin/watchdog -s -v -c /etc/watchdog.conf

Jun 24 01:23:41 odroid64 systemd[1]: Started watchdog daemon.
Jun 24 01:23:41 odroid64 watchdog[1405]: ping: no machine to check
Jun 24 01:23:41 odroid64 watchdog[1405]: file: no file to check
Jun 24 01:23:41 odroid64 watchdog[1405]: pidfile: /var/run/rsyslogd.pid
Jun 24 01:23:41 odroid64 watchdog[1405]: interface: no interface to check
Jun 24 01:23:41 odroid64 watchdog[1405]: temperature: no sensors to check
Jun 24 01:23:41 odroid64 watchdog[1405]: test=none(0) repair=none(0) alive=/dev/
Jun 24 01:23:41 odroid64 watchdog[1405]: watchdog now set to 30 seconds
Jun 24 01:23:41 odroid64 watchdog[1405]: hardware watchdog identity: Meson GXBB
Jun 24 01:23:41 odroid64 watchdog[1405]: was able to ping process 472 (/var/run/

Code: Select all

[sudo] password for odroid:
root@odroid64:~# kill -9 1405
root@odroid64:~# [  225.599297] watchdog watchdog0: watchdog did not stop!
root@odroid64:~#
root@odroid64:~#
root@odroid64:~#
on debugging further it seems like the internal ping of watchdog is still active and that prevents the reset and reboot.
so the watchdog keepalive process some how keep the watchdog device busy.

moon.linux
Posts: 1162
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english
Contact:

Re: C2 watchdog

Unread post by moon.linux » Fri Jun 24, 2016 3:47 pm

Their seem to lots of core changes in watchdog architecture to handle keepalive ping.
So it's hard to port back the changes to 3.14.x, so a simple test should work out well.

Code: Select all

/*
 * Watchdog Driver Test Program
 */

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <signal.h>
#include <sys/ioctl.h>
#include <linux/types.h>
#include <linux/watchdog.h>

int fd;

/*
 * This function simply sends an IOCTL to the driver, which in turn ticks
 * the PC Watchdog card to reset its internal timer so it doesn't trigger
 * a computer reset.
 */
static void keep_alive(void)
{
    int dummy;

    ioctl(fd, WDIOC_KEEPALIVE, &dummy);
}

/*
 * The main program.  Run the program with "-d" to disable the card,
 * or "-e" to enable the card.
 */

static void term(int sig)
{
    close(fd);
    fprintf(stderr, "Stopping watchdog ticks...\n");
    exit(0);
}

int main(int argc, char *argv[])
{
    int flags;
    unsigned int ping_rate = 1;

    fd = open("/dev/watchdog", O_WRONLY);

    if (fd == -1) {
	fprintf(stderr, "Watchdog device not enabled.\n");
	fflush(stderr);
	exit(-1);
    }

    if (argc > 1) {
	if (!strncasecmp(argv[1], "-d", 2)) {
	    flags = WDIOS_DISABLECARD;
	    ioctl(fd, WDIOC_SETOPTIONS, &flags);
	    fprintf(stderr, "Watchdog card disabled.\n");
	    fflush(stderr);
	    goto end;
	} else if (!strncasecmp(argv[1], "-e", 2)) {
	    flags = WDIOS_ENABLECARD;
	    ioctl(fd, WDIOC_SETOPTIONS, &flags);
	    fprintf(stderr, "Watchdog card enabled.\n");
	    fflush(stderr);
	    goto end;
	} else if (!strncasecmp(argv[1], "-t", 2) && argv[2]) {
	    flags = atoi(argv[2]);
	    ioctl(fd, WDIOC_SETTIMEOUT, &flags);
	    fprintf(stderr, "Watchdog timeout set to %u seconds.\n", flags);
	    fflush(stderr);
	    goto end;
	} else if (!strncasecmp(argv[1], "-p", 2) && argv[2]) {
	    ping_rate = strtoul(argv[2], NULL, 0);
	    fprintf(stderr, "Watchdog ping rate set to %u seconds.\n", ping_rate);
	    fflush(stderr);
	} else {
	    fprintf(stderr, "-d to disable, -e to enable, -t <n> to set " \
		"the timeout,\n-p <n> to set the ping rate, and \n");
	    fprintf(stderr, "run by itself to tick the card.\n");
	    fflush(stderr);
	    goto end;
	}
    }

    fprintf(stderr, "Watchdog Ticking Away!\n");
    fflush(stderr);

    signal(SIGINT, term);

    while(1) {
	keep_alive();
	sleep(ping_rate);
    }
end:
    close(fd);
    return 0;
}

Code: Select all

root@odroid64:~/./watchdog-test -p 10
Watchdog ping rate set to 10 seconds.
Watchdog Ticking Away!
On killing the watchdog process it triggers reboot.

Code: Select all

odroid@odroid64:~$ pgrep watchdog-test
1609
odroid@odroid64:~$ sudo kill -9 1609
[sudo] password for odroid:
[  466.482042] watchdog watchdog0: watchdog did not stop!
odroid@odroid64:~$ GXBB:BL1:08dafd:0a8993;FEAT:EDFC318C;POC:3;RCY:0;EMMC:800;NAND:81;SD:0;READ:0;CHK:0;
TE: 112222
no sdio debug board detected

BL2 Built : 11:44:26, Nov 25 2015.
gxb gfb13a3b-c2 - jcao@wonton

Board ID = 8
set vcck to 1100 mv
set vddee to 1050 mv
CPU clk: 1536MHz
DDR channel setting: DDR0 Rank0+1 same
DDR0: 2048MB(auto) @ 912MHz(2T)-13
DataBus test pass!
AddrBus test pass!
Load fip header from SD, src: 0x0000c200, des: 0x01400000, size: 0x000000b0
Load bl30 from SD, src: 0x00010200, des: 0x01000000, size: 0x00009ef0
Sending bl30........................................OK.
Run bl30...
Load bl301 from SD, src: 0x0001c200, des: 0x01000000, size: 0x000017c0

Bl4ckD0g
Posts: 48
Joined: Sat Apr 09, 2016 3:18 am
languages_spoken: english
ODROIDs: C2
Contact:

Re: C2 watchdog

Unread post by Bl4ckD0g » Fri Jul 01, 2016 10:18 am

Interesting.

Any estimate to have this working in the kernel?

Thanks!

moon.linux
Posts: 1162
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english
Contact:

Re: C2 watchdog

Unread post by moon.linux » Fri Jul 01, 2016 11:29 am

Bl4ckD0g wrote:Is there any plan to support the watchdog in the kernel?

It would help a lot to automatically reboot the C2 when we have hung states like the issue with the setting "nographics" set to 1:

http://forum.odroid.com/viewtopic.php?f=136&t=21520
Please let me know how to reproduce this hung state.
Ok I will set this option and try to test this kernel module.

Bl4ckD0g
Posts: 48
Joined: Sat Apr 09, 2016 3:18 am
languages_spoken: english
ODROIDs: C2
Contact:

Re: C2 watchdog

Unread post by Bl4ckD0g » Fri Jul 01, 2016 8:38 pm

moon.linux wrote:
Bl4ckD0g wrote:Is there any plan to support the watchdog in the kernel?

It would help a lot to automatically reboot the C2 when we have hung states like the issue with the setting "nographics" set to 1:

http://forum.odroid.com/viewtopic.php?f=136&t=21520
Please let me know how to reproduce this hung state.
Ok I will set this option and try to test this kernel module.
We would need to get the buggy boot.ini file/version.
I don't have it anymore, as @mdrjr has fixed the issue.

The repository (http://deb.odroid.in/c2/pool/main/b/bootini/) doesn't have the version anymore.
And I didn't find hardkernel's git for the boot.ini to be able to get it.

mscdex
Posts: 74
Joined: Sun Jan 04, 2015 11:09 am
languages_spoken: english
ODROIDs: ODROID C2
Contact:

Re: C2 watchdog

Unread post by mscdex » Fri Jul 01, 2016 9:04 pm

Bl4ckD0g wrote:
We would need to get the buggy boot.ini file/version.
I don't have it anymore, as @mdrjr has fixed the issue.

The repository (http://deb.odroid.in/c2/pool/main/b/bootini/) doesn't have the version anymore.
And I didn't find hardkernel's git for the boot.ini to be able to get it.
You tried this repo?

Bl4ckD0g
Posts: 48
Joined: Sat Apr 09, 2016 3:18 am
languages_spoken: english
ODROIDs: C2
Contact:

Re: C2 watchdog

Unread post by Bl4ckD0g » Mon Jul 04, 2016 8:22 pm

mscdex wrote:
Bl4ckD0g wrote:
We would need to get the buggy boot.ini file/version.
I don't have it anymore, as @mdrjr has fixed the issue.

The repository (http://deb.odroid.in/c2/pool/main/b/bootini/) doesn't have the version anymore.
And I didn't find hardkernel's git for the boot.ini to be able to get it.
You tried this repo?
Thanks @mscdex!

So it seems that we could reproduce the issue by using the boot.ini from bellow:

https://github.com/mdrjr/c2_bootini/blo ... b/boot.ini

Bl4ckD0g
Posts: 48
Joined: Sat Apr 09, 2016 3:18 am
languages_spoken: english
ODROIDs: C2
Contact:

Re: C2 watchdog

Unread post by Bl4ckD0g » Tue Jul 12, 2016 9:15 pm

This is new:
http://odroid.com/dokuwiki/doku.php?id= ... hdog_timer

How do I enable the gxbb_wdt driver/module?

I didn't find it anywhere.
Is it ready for "production" use?

moon.linux
Posts: 1162
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english
Contact:

Re: C2 watchdog

Unread post by moon.linux » Tue Jul 12, 2016 10:26 pm

sorry I am working on that please let me update the latest driver and it need some more testing.

Bl4ckD0g
Posts: 48
Joined: Sat Apr 09, 2016 3:18 am
languages_spoken: english
ODROIDs: C2
Contact:

Re: C2 watchdog

Unread post by Bl4ckD0g » Tue Jul 12, 2016 10:54 pm

Sure. No rush.

I was wondering if it was production ready as it's already in the wiki and I couldn't make it work.
Even checking/trying to compile the kernel I didn't see the driver for it in there.

Please let me know about any update.

And thanks!

moon.linux
Posts: 1162
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english
Contact:

Re: C2 watchdog

Unread post by moon.linux » Thu Jul 14, 2016 5:18 am

Please find the patches for watchdog driver attached.

simple test for watchdog reset.

Code: Select all

odroid@odroid64:~$ sudo -s
[sudo] password for odroid:
root@odroid64:~#  echo 5 > /dev/watchdog
[  152.334799] watchdog watchdog0: watchdog did not stop!
root@odroid64:~# GXBB:BL1:08dafd:0a8993;FEAT:EDFC318C;POC:3;RCY:0;EMMC:800;NAND:81;SD:0;READ:0;CHK:0;
TE: 112568
no sdio debug board detected

BL2 Built : 11:44:26, Nov 25 2015.
gxb gfb13a3b-c2 - jcao@wonton
You can configure watchdog using following link.
http://odroid.com/dokuwiki/doku.php?id= ... hdog_timer

After reboot check the status of the watchdog driver.

Code: Select all

odroid@odroid64:~$ service watchdog status
● watchdog.service - watchdog daemon
   Loaded: loaded (/lib/systemd/system/watchdog.service; static; vendor preset:
   Active: active (running) since Thu 2016-02-11 11:28:05 EST; 5 months 0 days a
  Process: 1825 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/wa
  Process: 1821 ExecStartPre=/bin/sh -c [ -z "${watchdog_module}" ] || [ "${watc
 Main PID: 1827 (watchdog)
   CGroup: /system.slice/watchdog.service
           └─1827 /usr/sbin/watchdog -s -v -c /etc/watchdog.conf

Jul 13 15:53:12 odroid64 watchdog[1827]: still alive after 1385 interval(s)
Jul 13 15:53:12 odroid64 watchdog[1827]: was able to ping process 1532 (/var/run
Jul 13 15:53:17 odroid64 watchdog[1827]: still alive after 1386 interval(s)
Jul 13 15:53:17 odroid64 watchdog[1827]: was able to ping process 1532 (/var/run
Jul 13 15:53:22 odroid64 watchdog[1827]: still alive after 1387 interval(s)
Jul 13 15:53:22 odroid64 watchdog[1827]: was able to ping process 1532 (/var/run
Jul 13 15:53:27 odroid64 watchdog[1827]: still alive after 1388 interval(s)
Jul 13 15:53:27 odroid64 watchdog[1827]: was able to ping process 1532 (/var/run
Jul 13 15:53:32 odroid64 watchdog[1827]: still alive after 1389 interval(s)
Jul 13 15:53:32 odroid64 watchdog[1827]: was able to ping process 1532 (/var/run
Issue with watchog service is that it's not able to remove the loaded driver if we trigger killing of watchdog process.
The module gxbb_wdt is not removed by watchdog exit status.

Code: Select all

odroid@odroid64:~$ sudo kill -9 1805
[sudo] password for odroid:
odroid@odroid64:~$ [   78.201288] watchdog watchdog0: watchdog did not stop!

odroid@odroid64:~$
odroid@odroid64:~$
odroid@odroid64:~$ service watchdog status
● watchdog.service - watchdog daemon
   Loaded: loaded (/lib/systemd/system/watchdog.service; static; vendor preset:
   Active: inactive (dead) since Thu 2016-02-11 11:29:08 EST; 8s ago
  Process: 2358 ExecStopPost=/bin/sh -c [ $run_wd_keepalive != 1 ] || false (cod
  Process: 1802 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/wa
  Process: 1799 ExecStartPre=/bin/sh -c [ -z "${watchdog_module}" ] || [ "${watc
 Main PID: 1805 (code=killed, signal=KILL)

Feb 11 11:28:55 odroid64 watchdog[1805]: was able to ping process 1597 (/var/run
Feb 11 11:29:00 odroid64 watchdog[1805]: still alive after 11 interval(s)
Feb 11 11:29:00 odroid64 watchdog[1805]: was able to ping process 1597 (/var/run
Feb 11 11:29:05 odroid64 watchdog[1805]: still alive after 12 interval(s)
Feb 11 11:29:05 odroid64 watchdog[1805]: was able to ping process 1597 (/var/run
Feb 11 11:29:08 odroid64 systemd[1]: watchdog.service: Main process exited, code
Feb 11 11:29:08 odroid64 systemd[1]: watchdog.service: Control process exited, c
Feb 11 11:29:08 odroid64 systemd[1]: watchdog.service: Unit entered failed state
Feb 11 11:29:08 odroid64 systemd[1]: watchdog.service: Triggering OnFailure= dep
Feb 11 11:29:08 odroid64 systemd[1]: watchdog.service: Failed with result 'exit-
odroid@odroid64:~$
odroid@odroid64:~$
odroid@odroid64:~$ sudo rmmod gxbb_wdt
[sudo] password for odroid:
odroid@odroid64:~$ GXBB:BL1:08dafd:0a8993;FEAT:EDFC318C;POC:3;RCY:0;EMMC:800;NAND:81;SD:0;READ:0;CHK:0;
TE: 112217
no sdio debug board detected

BL2 Built : 11:44:26, Nov 25 2015.
gxb gfb13a3b-c2 - jcao@wonton
Attachments
wd.tar.gz
watchdog driver.
(4.36 KiB) Downloaded 85 times

Bl4ckD0g
Posts: 48
Joined: Sat Apr 09, 2016 3:18 am
languages_spoken: english
ODROIDs: C2
Contact:

Re: C2 watchdog

Unread post by Bl4ckD0g » Thu Jul 14, 2016 10:19 pm

Hi moon.linux,

I just upgraded the kernel to 3.14.65-71, which brings the watchdog in.
Everything is set up, as described in the wiki page.
I was able to test it and it's doing what it's supposed to do :D

Hopefully it'll act in case of an intermittent hung (becoming rarer there days, as the C2 kernel is more mature).

Thank you very much for enabling and helping on this!
Really appreciated.

Regards,
Bl4ckD0g

JosArvio
Posts: 18
Joined: Wed Dec 07, 2016 3:42 pm
languages_spoken: english
ODROIDs: > 40
Contact:

Re: C2 watchdog

Unread post by JosArvio » Fri Jun 30, 2017 9:16 pm

Hi,

I configured C2 according http://odroid.com/dokuwiki/doku.php?id= ... hdog_timer without problems.
The watchdog service is running fine after reboot, but after running following 'pkill -9 watchdog' the watchdog service is dead.
Only after restarting the service it becomes alive, which defeats the purpose of a watchdog.

Same issue is described by moon.linux earlier in this post.
Anybody else experienced the same issue?
The C2 is configured with kernel 3.14.79-114. Packages are updated.
Same happened with a second C2.

Code: Select all

Using username "root".
Welcome to Ubuntu 16.04.2 LTS (GNU/Linux 3.14.79-114 aarch64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

0 packages can be updated.
0 updates are security updates.

Last login: Thu Feb 11 11:28:21 2016
root@odroid64:~# service watchdog status
● watchdog.service - watchdog daemon
   Loaded: loaded (/lib/systemd/system/watchdog.service; static; vendor preset: enabled)
   Active: active (running) since Thu 2016-02-11 11:28:07 EST; 43s ago
  Process: 861 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/watchdog $watchdog_options (code=exited, status=0/SUCCESS)
  Process: 857 ExecStartPre=/bin/sh -c [ -z "${watchdog_module}" ] || [ "${watchdog_module}" = "none" ] || /sbin/modprobe $watchdog_module (code=exited, status=0/SUCCESS)
 Main PID: 864 (watchdog)
   CGroup: /system.slice/watchdog.service
           └─864 /usr/sbin/watchdog -s -v -c /etc/watchdog.conf

Feb 11 11:28:47 odroid64 watchdog[864]: still alive after 37 interval(s)
Feb 11 11:28:47 odroid64 watchdog[864]: was able to ping process 599 (/var/run/rsyslogd.pid).
Feb 11 11:28:48 odroid64 watchdog[864]: still alive after 38 interval(s)
Feb 11 11:28:48 odroid64 watchdog[864]: was able to ping process 599 (/var/run/rsyslogd.pid).
Feb 11 11:28:49 odroid64 watchdog[864]: still alive after 39 interval(s)
Feb 11 11:28:49 odroid64 watchdog[864]: was able to ping process 599 (/var/run/rsyslogd.pid).
Feb 11 11:28:50 odroid64 watchdog[864]: still alive after 40 interval(s)
Feb 11 11:28:50 odroid64 watchdog[864]: was able to ping process 599 (/var/run/rsyslogd.pid).
Feb 11 11:28:51 odroid64 watchdog[864]: still alive after 41 interval(s)
Feb 11 11:28:51 odroid64 watchdog[864]: was able to ping process 599 (/var/run/rsyslogd.pid).
root@odroid64:~# pkill -9 watchdog
root@odroid64:~# service watchdog status
● watchdog.service - watchdog daemon
   Loaded: loaded (/lib/systemd/system/watchdog.service; static; vendor preset: enabled)
   Active: inactive (dead) since Thu 2016-02-11 11:29:07 EST; 4s ago
  Process: 1347 ExecStopPost=/bin/sh -c [ $run_wd_keepalive != 1 ] || false [color=#FF0000](code=exited, status=1/FAILURE)[/color]
  Process: 861 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/watchdog $watchdog_options (code=exited, status=0/SUCCESS)
  Process: 857 ExecStartPre=/bin/sh -c [ -z "${watchdog_module}" ] || [ "${watchdog_module}" = "none" ] || /sbin/modprobe $watchdog_module (code=exited, status=0/SUCCESS)
 Main PID: 864 (code=killed, signal=KILL)

Feb 11 11:29:04 odroid64 watchdog[864]: was able to ping process 599 (/var/run/rsyslogd.pid).
Feb 11 11:29:05 odroid64 watchdog[864]: still alive after 55 interval(s)
Feb 11 11:29:05 odroid64 watchdog[864]: was able to ping process 599 (/var/run/rsyslogd.pid).
Feb 11 11:29:06 odroid64 watchdog[864]: still alive after 56 interval(s)
Feb 11 11:29:06 odroid64 watchdog[864]: was able to ping process 599 (/var/run/rsyslogd.pid).
Feb 11 11:29:07 odroid64 systemd[1]: watchdog.service: Main process exited, code=killed, status=9/KILL
Feb 11 11:29:07 odroid64 systemd[1]: watchdog.service: Control process exited, code=exited status=1
Feb 11 11:29:07 odroid64 systemd[1]: watchdog.service: Unit entered failed state.
Feb 11 11:29:07 odroid64 systemd[1]: watchdog.service: Triggering OnFailure= dependencies.
Feb 11 11:29:07 odroid64 systemd[1]: watchdog.service: Failed with result 'exit-code'.

moon.linux
Posts: 1162
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english
Contact:

Re: C2 watchdog

Unread post by moon.linux » Thu Jul 06, 2017 3:15 pm

Hi JosArvio,

Sorry for the delay on this, Actually the watchdog module is bit complicated with the timer module and PMU reset.
When we set up a timer via watchdog daemon we are periodically reset the timer, once the timer expires the watchdog module will trigger are reset of board.

In the current scenario I think the timer is not able to set up correctly or the PMU event is not generates, I need some to to debug this.

Mean while could you find a testcase with could simulate the watchdog reset externally, when the watchdog driver is under usage of watchog demon.
We have not seen major kernel crash to trigger watchdog reset.

I have seen above behavior at on my Odroid C2.

User avatar
mad_ady
Posts: 5454
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, N1, H2, N2
Location: Bucharest, Romania
Contact:

Re: C2 watchdog

Unread post by mad_ady » Thu Jul 06, 2017 4:14 pm

@moon.linux: I too have noticed that the watchdog does not restart my system in some cases.
You can generate a kernel panic with:

Code: Select all

echo 1 > /proc/sys/kernel/sysrq
echo c > /proc/sysrq-trigger

JosArvio
Posts: 18
Joined: Wed Dec 07, 2016 3:42 pm
languages_spoken: english
ODROIDs: > 40
Contact:

Re: C2 watchdog

Unread post by JosArvio » Thu Jul 06, 2017 4:20 pm

Hi moon.linux,

Thanks for looking into this and reproducing our findings, much appreciated.

I did some more research.
After C2 is started the watchdog service is running fine using command "service watchdog status"
Commands I tried which leave watchdog service 'inactive (dead)':
service watchdog restart
halt
pkill -9 watchdog

However I can confirm the watchdog is working.
I changed the /etc/watchdog.conf and enabled 'ping' to a machine. Once the pinged machine is no longer on the network the watchdog triggered a restart. Which is good.

So it looks like management of the watchdog service is not quite right.
When watchdog is dead and you look at the status it shows an error: (code=exited, status=1/FAILURE)
I don't know what this means.
Attachments
StatusWatchdogC2.png
StatusWatchdogC2.png (178.19 KiB) Viewed 11310 times

moon.linux
Posts: 1162
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english
Contact:

Re: C2 watchdog

Unread post by moon.linux » Thu Jul 06, 2017 6:45 pm

Thank for your input. I will checkout.

paulcrawford
Posts: 120
Joined: Tue May 10, 2016 11:51 am
languages_spoken: english
ODROIDs: odroid c2
Contact:

Re: C2 watchdog

Unread post by paulcrawford » Sat Jul 15, 2017 10:51 am

@odroid has stated that the WDT in the C2 is not reliable:viewtopic.php?f=140&t=24468

I have used an external watchdog implemented with an ATMega328P. I used two gpio pins on the C2, one to send the pulse to reset the WDT and one to indicate that the WDT is active. The ATMega signal, going back the C2 if the WDT is not reset, puts 5V on pin 1 of J4 on the C2. I have tested this arrangement and it works perfectly. The ATMega was programmed with a very short C program which was compiled and uploaded with AVR Dude and a USBTiny programmer.

linuxest
Posts: 43
Joined: Tue Nov 08, 2016 6:35 pm
languages_spoken: english
ODROIDs: 1 x HC1 and 2 x C2s with HiFi-Shield 2
And some RPi3 boards
Contact:

Re: C2 watchdog

Unread post by linuxest » Sat Jul 15, 2017 11:12 am

@odroid made a similar hw solution on XU4 too. viewtopic.php?f=98&t=26543#p184993

But I hope @moon.linux will find a pure software solution for C2. I don't want to add a bulky hardware watch-dog board even it is more reliable.

paulcrawford
Posts: 120
Joined: Tue May 10, 2016 11:51 am
languages_spoken: english
ODROIDs: odroid c2
Contact:

Re: C2 watchdog

Unread post by paulcrawford » Sat Jul 15, 2017 8:04 pm

I understand about the "bulky hardware watch-dog board" but in my case I needed the ATMega for another purpose anyway so essentially the WDT came for free...

moon.linux
Posts: 1162
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english
Contact:

Re: C2 watchdog

Unread post by moon.linux » Tue Jul 18, 2017 4:33 pm

@paulcrawford
Sorry for some late response,I did some quick testing today and it seen to work for me.
The ping test failed and I got the board to reboot using watchdog reset.

Watchdog module configuration.

Code: Select all

odroid@odroid64:~$ cat /etc/default/watchdog
# Start watchdog at boot time? 0 or 1
run_watchdog=1
# Start wd_keepalive after stopping watchdog? 0 or 1
run_wd_keepalive=1
# Load module before starting watchdog
watchdog_module="gxbb_wdt"
# Specify additional watchdog options here (see manpage).
watchdog_options="-s -v -c /etc/watchdog.conf"
Watchdog configuration

Code: Select all

odroid@odroid64:~$ cat /etc/watchdog.conf
ping                    = 10.0.0.101
#ping                   = 172.26.1.255
interface               = eth0
file                    = /var/log/watchdog/watchdog
#change                 = 1407

# Uncomment to enable test. Setting one of these values to '0' disables it.
# These values will hopefully never reboot your machine during normal use
# (if your machine is really hung, the loadavg will go much higher than 25)
#max-load-1             = 24
#max-load-5             = 18
#max-load-15            = 12

# Note that this is the number of pages!
# To get the real size, check how large the pagesize is on your machine.
#min-memory             = 1
#allocatable-memory     = 1

#repair-binary          = /usr/sbin/repair
#repair-timeout         =
#test-binary            =
#test-timeout           =

watchdog-device = /dev/watchdog

# Defaults compiled into the binary
#temperature-device     =
#max-temperature        = 120

# Defaults compiled into the binary
admin                   = root
interval                = 1
logtick                = 1
log-dir         = /var/log/watchdog

# This greatly decreases the chance that watchdog won't be scheduled before
# your machine is really loaded
realtime                = yes
priority                = 1

# Check if rsyslogd is still running by enabling the following line
pidfile         = /var/run/rsyslogd.pid

watchdog-timeout        = 15
Ping test failed to the IP

Code: Select all

odroid@odroid64:~$ ping 10.0.0.101
PING 10.0.0.101 (10.0.0.101) 56(84) bytes of data.
From 10.0.0.138 icmp_seq=1 Destination Host Unreachable
From 10.0.0.138 icmp_seq=2 Destination Host Unreachable
From 10.0.0.138 icmp_seq=3 Destination Host Unreachable
From 10.0.0.138 icmp_seq=4 Destination Host Unreachable
From 10.0.0.138 icmp_seq=5 Destination Host Unreachable
From 10.0.0.138 icmp_seq=6 Destination Host Unreachable
From 10.0.0.138 icmp_seq=7 Destination Host Unreachable
Watchdog service status is not enabled at the start.

Code: Select all

odroid@odroid64:~$ service watchdog status
● watchdog.service - watchdog daemon
   Loaded: loaded (/lib/systemd/system/watchdog.service; static; vendor preset:
   Active: inactive (dead)
odroid@odroid64:~$
Once we enable the watchdog service I immediate get an watchdog reset to reboot.

Code: Select all

odroid@odroid64:~$ sudo service watchdog restart
[sudo] password for odroid:
odroid@odroid64:~$
odroid@odroid64:~$ sudo service watchdog status
● watchdog.service - watchdog daemon
   Loaded: loaded (/lib/systemd/system/watchdog.service; static; vendor preset:
   Active: active (running) since Tue 2017-07-18 03:24:46 EDT; 5s ago
  Process: 2138 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/wa
  Process: 2134 ExecStartPre=/bin/sh -c [ -z "${watchdog_module}" ] || [ "${watc
 Main PID: 2140 (watchdog)
   CGroup: /system.slice/watchdog.service
           └─2140 /usr/sbin/watchdog -s -v -c /etc/watchdog.conf

Jul 18 03:24:46 odroid64 watchdog[2140]: device eth0 received 80277 bytes
Jul 18 03:24:46 odroid64 watchdog[2140]: ping select timeout =  1.000000 seconds
Jul 18 03:24:47 odroid64 watchdog[2140]: ping select timeout =  1.000000 seconds
Jul 18 03:24:48 odroid64 watchdog[2140]: ping select timeout =  1.000000 seconds
Jul 18 03:24:49 odroid64 watchdog[2140]: ping select timeout =  0.008928 seconds
Jul 18 03:24:49 odroid64 watchdog[2140]: ping select timeout =  0.008864 seconds
Jul 18 03:24:49 odroid64 watchdog[2140]: ping select timeout =  0.008844 seconds
Jul 18 03:24:49 odroid64 watchdog[2140]: no response from ping (target: 10.0.0.1
Jul 18 03:24:49 odroid64 watchdog[2140]: /usr/lib/sendmail does not exist or is
Jul 18 03:24:49 odroid64 watchdog[2140]: shutting down the system because of err
odroid@odroid64:~$ [  619.294438] watchdog watchdog0: watchdog did not stop!
GXBB:BL1:08dafd:0a8993;FEAT:EDFC318C;POC:3;RCY:0;EMMC:800;NAND:81;SD:0;READ:0;CHK:0;
TE: 99064
no sdio debug board detected

BL2 Built : 11:44:26, Nov 25 2015.
gxb gfb13a3b-c2 - jcao@wonton

Board ID = 8
set vcck to 1100 mv
set vddee to 1050 mv
CPU clk: 1536MHz
DDR channel setting: DDR0 Rank0+1 same
DDR0: 2048MB(auto) @ 912MHz(2T)-13
DataBus test pass!
AddrBus test pass!
Load fip header from SD, src: 0x0000c200, des: 0x01400000, size: 0x000000b0
Load bl30 from SD, src: 0x00010200, des: 0x01000000, size: 0x00009ef0
Sending bl30........................................OK.
Run bl30...
Load bl301 from SD, src: 0x0001c200, des: 0x01000000, size: 0x000018c0
Wait bl30...Done
Sending bl301.......OK.
Run bl301...
1 from SD, src: 0x00020200, des: 0x10100000, size: 0x00011130


--- UART initialized after reboot ---
[Reset cause: unknown]
[Image: unknown, amlogic_v1.1.3046-00db630-dirty 2016-08-31 09:24:14 tao.zeng@droid04]
bl30: check_permit, count is 1
bl30: check_permit: ok!
chipid:Load bl33 from SD, src: 0x00034200, des: 0x01000000, size: 0x00073510
 ef be ad de d f0 ad ba ef be ad de not ES chip
[0.213687 Inits done]
secure task start!
high task start!
low task start!
NOTICE:  BL3-1: v1.0(debug):4d2e34d
NOTICE:  BL3-1: Built : 17:08:35, Oct 29 2015
INFO:    BL3-1: Initializing runtime services
INFO:    BL3-1: Preparing for EL3 exit to normal world
INFO:    BL3-1: Next image address = 0x1000000
INFO:    BL3-1: Next image spsr = 0x3c9


U-Boot 2015.01-00189-g205c7b3 (Feb 27 2017 - 10:53:13)

DRAM:  2 GiB
Relocation Offset is: 76f32000
-------------------------------------------------
* Welcome to Hardkernel's ODROID-C2
-------------------------------------------------
CPU : AMLogic S905
S/N : HKC213254DFDA0AD
MAC : 00:1e:06:33:ed:2d
BID : HKC2211612
-------------------------------------------------
register usb cfg[1][0] = 0000000077f95058
register usb cfg[0][1] = 0000000077f95078
vpu detect type: 5
vpu clk_level = 7
set vpu clk: 666667000Hz, readback: 666660000Hz(0x300)
MMC:   aml_priv->desc_buf = 0x0000000073f2ad30
aml_priv->desc_buf = 0x0000000073f2cec0
SDIO Port B: 0, SDIO Port C: 1
ret = 1 .[mmc_init] mmc init success
In:    serial
Out:   serial
Err:   serial
----------------------------------
MMC Size : 32 GB
----------------------------------
reading boot-logo.bmp.gz
** Unable to read file boot-logo.bmp.gz **
reading boot-logo.bmp
** Unable to read file boot-logo.bmp **
movi: the partiton 'logo' is reading...

MMC read: dev # 0, block # 61024, count 2048 ... 2048 blocks read: OK
hpd_state=1
[CANVAS]addr=0x3f800000 width=3840, height=1440

set hdmitx VIC = 16
hdmitx phy setting done
set hdmitx VIC = 16
hdmitx phy setting done
Error: Bad gzipped data
There is no valid bmp file at the given address
Net:   Meson_Ethernet
Hit [Enter] key twice to stop autoboot:  0
reading boot.ini
6790 bytes read in 3 ms (2.2 MiB/s)
cfgload: applying boot.ini...
cfgload: setenv display_autodetect "true"
cfgload: setenv m "1080p60hz" # Progressive 60Hz
cfgload: setenv m_bpp "32"
cfgload: setenv hpd "true"
cfgload: setenv monitor_onoff "false" # true or false
cfgload: setenv nographics "0"
cfgload: setenv mesontimer "1"
cfgload: setenv disableuhs "false"
cfgload: setenv mmc_removable "true"
cfgload: setenv usbmulticam "false"
cfgload: setenv disable_vu7 "false"
cfgload: setenv condev "console=ttyS0,115200n8 console=tty0"   # on both
cfgload: setenv maxcpus "4"
cfgload: setenv max_freq "1536"  # 1.536GHz
cfgload: if test "${display_autodetect}" = "true"; then usb pwren; hdmitx edid; fi
USB0:   dwc_usb driver version: 2.94 6-June-2012
USB (1) peri reg base: c0000020
USB (1) use clock source: XTAL input, div: 1
USB (1) base addr: 0xc9100000
Force id mode: Host
dwc_otg: Highspeed device found !

..............................
..............................
..............................
..............................
..............................
..............................
..............................
..............................
..............................
..............................
..............................
..............................
..............................
..............................
..............................
..............................
edid extension block number : 1
No header found
Manufacturer: @@@ Model 0 Serial Number 0
EDID version: 0.0
Analog display, Input voltage level: 0.7/0.3 V
Sync:
Established timings supported:
Standard timings supported:
non-conformant standard timing (0 horiz)
non-conformant standard timing (0 horiz)
non-conformant standard timing (0 horiz)
non-conformant standard timing (0 horiz)
non-conformant standard timing (0 horiz)
non-conformant standard timing (0 horiz)
non-conformant standard timing (0 horiz)
non-conformant standard timing (0 horiz)
Manufacturer-specified data, tag 0
Manufacturer-specified data, tag 0
Manufacturer-specified data, tag 0
Manufacturer-specified data, tag 0
Checksum: 0x0 (valid)
bestmode is 1080p60hz, IEEEOUI 0x000000
When I set up the ping ip to router default gateway.

Code: Select all

odroid@odroid64:~$ ping 10.0.0.1
PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data.
64 bytes from 10.0.0.1: icmp_seq=1 ttl=64 time=0.784 ms
64 bytes from 10.0.0.1: icmp_seq=2 ttl=64 time=0.823 ms
64 bytes from 10.0.0.1: icmp_seq=3 ttl=64 time=0.807 ms
^C
--- 10.0.0.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1998ms
rtt min/avg/max/mdev = 0.784/0.804/0.823/0.036 ms
odroid@odroid64:~$
Watchdog service seen to be stable.

Code: Select all

root@odroid64:~# sudo service watchdog status
● watchdog.service - watchdog daemon
   Loaded: loaded (/lib/systemd/system/watchdog.service; static; vendor preset:
   Active: active (running) since Tue 2017-07-18 03:58:34 EDT; 4min 47s ago
  Process: 3037 ExecStopPost=/bin/sh -c [ $run_wd_keepalive != 1 ] || false (cod
  Process: 3107 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/wa
  Process: 3103 ExecStartPre=/bin/sh -c [ -z "${watchdog_module}" ] || [ "${watc
 Main PID: 3109 (watchdog)
   CGroup: /system.slice/watchdog.service
           └─3109 /usr/sbin/watchdog -s -v -c /etc/watchdog.conf

Jul 18 04:03:20 odroid64 watchdog[3109]: still alive after 282 interval(s)
Jul 18 04:03:20 odroid64 watchdog[3109]: was able to ping process 543 (/var/run/
Jul 18 04:03:20 odroid64 watchdog[3109]: device eth0 received 259510 bytes
Jul 18 04:03:20 odroid64 watchdog[3109]: ping select timeout =  1.000000 seconds
Jul 18 04:03:20 odroid64 watchdog[3109]: got answer from target 10.0.0.1
Jul 18 04:03:21 odroid64 watchdog[3109]: still alive after 283 interval(s)
Jul 18 04:03:21 odroid64 watchdog[3109]: was able to ping process 543 (/var/run/
Jul 18 04:03:21 odroid64 watchdog[3109]: device eth0 received 259668 bytes
Jul 18 04:03:21 odroid64 watchdog[3109]: ping select timeout =  0.999999 seconds
Jul 18 04:03:21 odroid64 watchdog[3109]: got answer from target 10.0.0.1
Last edited by moon.linux on Tue Jul 18, 2017 5:04 pm, edited 1 time in total.

JosArvio
Posts: 18
Joined: Wed Dec 07, 2016 3:42 pm
languages_spoken: english
ODROIDs: > 40
Contact:

Re: C2 watchdog

Unread post by JosArvio » Tue Jul 18, 2017 4:50 pm

Hi moon.linux,

Your observations are the same as I found, the C2 watchdog is triggered by ping.
However in the instructions to setup watchdog, several tests are mentioned, which are not working. Following commands leave the watchdog in a 'dead' state.
service watchdog restart
halt
pkill -9 watchdog

Do you have a solution for this?

moon.linux
Posts: 1162
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english
Contact:

Re: C2 watchdog

Unread post by moon.linux » Tue Jul 18, 2017 5:32 pm

Yes I observer this, I am looking into this.

moon.linux
Posts: 1162
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english
Contact:

Re: C2 watchdog

Unread post by moon.linux » Wed Jul 19, 2017 2:13 am

It looks like the watchdog module is not getting unregistered on watchdog reset.

Code: Select all

odroid@odroid64:~$ sudo service watchdog status
● watchdog.service - watchdog daemon
   Loaded: loaded (/lib/systemd/system/watchdog.service; static; vendor preset:
   Active: inactive (dead)
odroid@odroid64:~$ sudo service watchdog restart
odroid@odroid64:~$ sudo service watchdog status
● watchdog.service - watchdog daemon
   Loaded: loaded (/lib/systemd/system/watchdog.service; static; vendor preset:
   Active: active (running) since Tue 2017-07-18 11:49:21 EDT; 2s ago
  Process: 1902 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/wa
  Process: 1898 ExecStartPre=/bin/sh -c [ -z "${watchdog_module}" ] || [ "${watc
 Main PID: 1904 (watchdog)
   CGroup: /system.slice/watchdog.service
           └─1904 /usr/sbin/watchdog -s -v -c /etc/watchdog.conf

Jul 18 11:49:22 odroid64 watchdog[1904]: still alive after 1 interval(s)
Jul 18 11:49:22 odroid64 watchdog[1904]: was able to ping process 507 (/var/run/
Jul 18 11:49:22 odroid64 watchdog[1904]: device eth0 received 16490 bytes
Jul 18 11:49:22 odroid64 watchdog[1904]: ping select timeout =  0.999999 seconds
Jul 18 11:49:22 odroid64 watchdog[1904]: got answer from target 10.0.0.1
Jul 18 11:49:23 odroid64 watchdog[1904]: still alive after 2 interval(s)
Jul 18 11:49:23 odroid64 watchdog[1904]: was able to ping process 507 (/var/run/
Jul 18 11:49:23 odroid64 watchdog[1904]: device eth0 received 17656 bytes
Jul 18 11:49:23 odroid64 watchdog[1904]: ping select timeout =  1.000000 seconds
Jul 18 11:49:23 odroid64 watchdog[1904]: got answer from target 10.0.0.1
odroid@odroid64:~$ sudo service watchdog status
● watchdog.service - watchdog daemon
   Loaded: loaded (/lib/systemd/system/watchdog.service; static; vendor preset:
   Active: active (running) since Tue 2017-07-18 11:49:21 EDT; 14s ago
  Process: 1902 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/wa
  Process: 1898 ExecStartPre=/bin/sh -c [ -z "${watchdog_module}" ] || [ "${watc
 Main PID: 1904 (watchdog)
   CGroup: /system.slice/watchdog.service
           └─1904 /usr/sbin/watchdog -s -v -c /etc/watchdog.conf

Jul 18 11:49:34 odroid64 watchdog[1904]: still alive after 13 interval(s)
Jul 18 11:49:34 odroid64 watchdog[1904]: was able to ping process 507 (/var/run/
Jul 18 11:49:34 odroid64 watchdog[1904]: device eth0 received 19353 bytes
Jul 18 11:49:34 odroid64 watchdog[1904]: ping select timeout =  0.999999 seconds
Jul 18 11:49:34 odroid64 watchdog[1904]: got answer from target 10.0.0.1
Jul 18 11:49:35 odroid64 watchdog[1904]: still alive after 14 interval(s)
Jul 18 11:49:35 odroid64 watchdog[1904]: was able to ping process 507 (/var/run/
Jul 18 11:49:35 odroid64 watchdog[1904]: device eth0 received 19451 bytes
Jul 18 11:49:35 odroid64 watchdog[1904]: ping select timeout =  0.999999 seconds
Jul 18 11:49:35 odroid64 watchdog[1904]: got answer from target 10.0.0.1
odroid@odroid64:~$
odroid@odroid64:~$ s
root@odroid64:~# pkill -9 watchdog
root@odroid64:~# [  167.694510] watchdog watchdog0: watchdog did not stop!

root@odroid64:~# lsmod
Module                  Size  Used by
fuse                   89333  3
ir_lirc_codec           4814  3
ir_mce_kbd_decoder      3381  0
ir_sanyo_decoder        1668  0
ir_sony_decoder         1698  0
ir_jvc_decoder          1688  0
ir_rc5_decoder          1605  0
ir_rc6_decoder          2021  0
lirc_dev               10631  1 ir_lirc_codec
ir_nec_decoder          1824  0
meson_ir                3404  0
zram                   18336  4
lz4_decompress          2704  1 zram
lz4_compress            2865  1 zram
gxbb_wdt                3427  0
meson_gpiomem           4024  0
ipv6                  376636  48
autofs4                28111  2
root@odroid64:~# rmmod gxbb_wdt
root@odroid64:~# lsmod
Module                  Size  Used by
fuse                   89333  3
ir_lirc_codec           4814  3
ir_mce_kbd_decoder      3381  0
ir_sanyo_decoder        1668  0
ir_sony_decoder         1698  0
ir_jvc_decoder          1688  0
ir_rc5_decoder          1605  0
ir_rc6_decoder          2021  0
lirc_dev               10631  1 ir_lirc_codec
ir_nec_decoder          1824  0
meson_ir                3404  0
zram                   18336  4
lz4_decompress          2704  1 zram
lz4_compress            2865  1 zram
meson_gpiomem           4024  0
ipv6                  376636  48
autofs4                28111  2
root@odroid64:~# GXBB:BL1:08dafd:0a8993;FEAT:EDFC318C;POC:3;RCY:0;EMMC:800;NAND:81;SD:0;READ:0;CHK:0;
TE: 99108
no sdio debug board detected

BL2 Built : 11:44:26, Nov 25 2015.
gxb gfb13a3b-c2 - jcao@wonton

Board ID = 8
set vcck to 1100 mv
set vddee to 1050 mv
CPU clk: 1536MHz
DDR channel setting: DDR0 Rank0+1 same
DDR0: 2048MB(auto) @ 912MHz(2T)-13
DataBus test pass!
AddrBus test pass!
Load fip header from SD, src: 0x0000c200, des: 0x01400000, size: 0x000000b0
Load bl30 from SD, src: 0x00010200, des: 0x01000000, size: 0x00009ef0
Sending bl30........................................OK.
Run bl30...
Load bl301 from SD, src: 0x0001c200, des: 0x01000000, size: 0x000018c0
Wait bl30...Done
Sending bl301.......OK.
Run bl301...
1 from SD, src: 0x00020200, des: 0x10100000, size: 0x00011130


--- UART initialized after reboot ---
[Reset cause: unknown]
[Image: unknown, amlogic_v1.1.3046-00db630-dirty 2016-08-31 09:24:14 tao.zeng@droid04]
bl30: check_permit, count is 1
bl30: check_permit: ok!
chipid:Load bl33 from SD, src: 0x00034200, des: 0x01000000, size: 0x00073510
 ef be ad de d f0 ad ba ef be ad de not ES chip
[0.213740 Inits done]
secure task start!
high task start!
low task start!
NOTICE:  BL3-1: v1.0(debug):4d2e34d
NOTICE:  BL3-1: Built : 17:08:35, Oct 29 2015
INFO:    BL3-1: Initializing runtime services
INFO:    BL3-1: Preparing for EL3 exit to normal world
INFO:    BL3-1: Next image address = 0x1000000
INFO:    BL3-1: Next image spsr = 0x3c9


U-Boot 2015.01-00189-g205c7b3 (Feb 27 2017 - 10:53:13)

DRAM:  2 GiB
Relocation Offset is: 76f32000
-------------------------------------------------
* Welcome to Hardkernel's ODROID-C2
-------------------------------------------------
CPU : AMLogic S905
S/N : HKC213254DFDA0AD
MAC : 00:1e:06:33:ed:2d
BID : HKC2211612
-------------------------------------------------
register usb cfg[1][0] = 0000000077f95058

moon.linux
Posts: 1162
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english
Contact:

Re: C2 watchdog

Unread post by moon.linux » Wed Jul 19, 2017 3:51 am

judging by the watchdog service error it seen to me watchdog service error.

https://bugs.launchpad.net/ubuntu/+sour ... ug/1535854

Code: Select all

Feb 11 11:29:17 odroid64 systemd[1]: watchdog.service: Failed with result 'signa
Warning: watchdog.service changed on disk. Run 'systemctl daemon-reload' to relo
root@odroid64:~# sudo service watchdog status
● watchdog.service - watchdog daemon
   Loaded: loaded (/lib/systemd/system/watchdog.service; static; vendor preset:
   Active: inactive (dead) since Thu 2016-02-11 11:29:17 EST; 1 years 5 months a
  Process: 1477 ExecStopPost=/bin/sh -c [ $run_wd_keepalive != 1 ] || false (cod
  Process: 1470 ExecStop=/bin/sh -c exec /sbin/rmmod gxbb_wdt (code=exited, stat
  Process: 1373 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/wa
  Process: 1370 ExecStartPre=/bin/sh -c [ -z "${watchdog_module}" ] || [ "${watc
 Main PID: 1376 (code=killed, signal=KILL)

Feb 11 11:29:15 odroid64 watchdog[1376]: got answer from target 10.0.0.1
Feb 11 11:29:16 odroid64 watchdog[1376]: still alive after 16 interval(s)
Feb 11 11:29:16 odroid64 watchdog[1376]: was able to ping process 566 (/var/run/
Feb 11 11:29:16 odroid64 watchdog[1376]: device eth0 received 13284 bytes
Feb 11 11:29:16 odroid64 watchdog[1376]: ping select timeout =  0.999999 seconds
Feb 11 11:29:16 odroid64 watchdog[1376]: got answer from target 10.0.0.1
Feb 11 11:29:17 odroid64 systemd[1]: watchdog.service: Main process exited, code
Feb 11 11:29:17 odroid64 systemd[1]: watchdog.service: Unit entered failed state
Feb 11 11:29:17 odroid64 systemd[1]: watchdog.service: Triggering OnFailure= dep
Feb 11 11:29:17 odroid64 systemd[1]: watchdog.service: Failed with result 'signa
Warning: watchdog.service changed on disk. Run 'systemctl daemon-reload' to relo
I tried to add ExecStop to the service but is also failed to execute.

Code: Select all

odroid@odroid64:~$ cat /lib/systemd/system/watchdog.service
[Unit]
Description=watchdog daemon
Conflicts=wd_keepalive.service
After=multi-user.target
OnFailure=wd_keepalive.service

[Service]
Type=forking
EnvironmentFile=/etc/default/watchdog
ExecStartPre=/bin/sh -c '[ -z "${watchdog_module}" ] || [ "${watchdog_module}" = "none" ] || /sbin/modprobe $watchdog_module'
ExecStart=/bin/sh -c '[ $run_watchdog != 1 ] || exec /usr/sbin/watchdog $watchdog_options'
ExecStopPost=/bin/sh -c '[ $run_wd_keepalive != 1 ] || false'
ExecStop=/bin/sh -c 'exec /sbin/rmmod gxbb_wdt'

[Install]

User avatar
mad_ady
Posts: 5454
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, N1, H2, N2
Location: Bucharest, Romania
Contact:

Re: C2 watchdog

Unread post by mad_ady » Wed Jul 19, 2017 3:23 pm

@moon.linux: You should try OnFailure or FailureAction, so it's executed only when the service is stopped without a clean exist code: https://serverfault.com/questions/76998 ... f-graceful

moon.linux
Posts: 1162
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english
Contact:

Re: C2 watchdog

Unread post by moon.linux » Wed Jul 19, 2017 10:52 pm

No this did not help. As far as I know the job of watchdog service is the unload the module on exit.

After watchdog is killed.

Code: Select all

root@odroid64:~# sudo service watchdog status
● watchdog.service - watchdog daemon
   Loaded: loaded (/lib/systemd/system/watchdog.service; disabled; vendor preset: enabled)
   Active: inactive (dead) since Wed 2017-07-19 09:48:09 EDT; 31s ago
  Process: 2113 ExecStopPost=/bin/sh -c exec /sbin/rmmod gxbb_wdt || [ $run_wd_keepalive != 1 ] || false (code=exited, status=0/SUCCESS)
  Process: 2096 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/watchdog $watchdog_options (code=exited, status=0/SUCCESS)
  Process: 2092 ExecStartPre=/bin/sh -c [ -z "${watchdog_module}" ] || [ "${watchdog_module}" = "none" ] || /sbin/modprobe $watchdog_module (code=exi
 Main PID: 2098 (code=killed, signal=KILL)

Jul 19 09:48:07 odroid64 watchdog[2098]: got answer from target 10.0.0.1
Jul 19 09:48:08 odroid64 watchdog[2098]: still alive after 20 interval(s)
Jul 19 09:48:08 odroid64 watchdog[2098]: was able to ping process 540 (/var/run/rsyslogd.pid).
Jul 19 09:48:08 odroid64 watchdog[2098]: device eth0 received 164933 bytes
Jul 19 09:48:08 odroid64 watchdog[2098]: ping select timeout =  1.000000 seconds
Jul 19 09:48:08 odroid64 watchdog[2098]: got answer from target 10.0.0.1
Jul 19 09:48:09 odroid64 systemd[1]: watchdog.service: Main process exited, code=killed, status=9/KILL
Jul 19 09:48:09 odroid64 systemd[1]: watchdog.service: Unit entered failed state.
Jul 19 09:48:09 odroid64 systemd[1]: watchdog.service: Triggering OnFailure= dependencies.
Jul 19 09:48:09 odroid64 systemd[1]: watchdog.service: Failed with result 'signal'.

JosArvio
Posts: 18
Joined: Wed Dec 07, 2016 3:42 pm
languages_spoken: english
ODROIDs: > 40
Contact:

Re: C2 watchdog

Unread post by JosArvio » Tue Oct 03, 2017 7:17 am

Hi Moon.Linux,

Any update about this topic? The watchdog functionality for C2 is not quite the same as for a C1. We want to start shipping C2 to our customers, but this watchdog issue prevents us doing it.

Jos

User avatar
odroid
Site Admin
Posts: 30050
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Contact:

Re: C2 watchdog

Unread post by odroid » Tue Oct 03, 2017 12:46 pm


moon.linux
Posts: 1162
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english
Contact:

Re: C2 watchdog

Unread post by moon.linux » Tue Oct 03, 2017 2:04 pm

Watchdog reset can be tested by trigger while update the Magic SysRq key, which should confirm that the when kernel panic happen it will trigger reboot.

https://wiki.odroid.com/odroid-c2/appli ... dog_daemon

themoonlitknight
Posts: 10
Joined: Mon Jul 07, 2014 2:55 am
languages_spoken: english
ODROIDs: C2, C1, U3
Contact:

Re: C2 watchdog

Unread post by themoonlitknight » Sun Jan 13, 2019 7:52 am

Hello, I have a Odroid C2 with Android 6.0.1. I need to use a hw watchdog but I can't find a way to use it on Android, since all the guides refer to Linux only. I looked into /lib/modules directory and there is nothing similar to gxbb_wdt. Also there is no /dev/watchdog path.

Is it possible to activate it? How?

Thank you!

User avatar
odroid
Site Admin
Posts: 30050
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Contact:

Re: C2 watchdog

Unread post by odroid » Mon Jan 14, 2019 11:44 am

@themoonlitknight,
Please open a new topic on C2 Android sub-forum.

Post Reply

Return to “General Topics”

Who is online

Users browsing this forum: No registered users and 2 guests