I made a Java GUI logger

[Regolo]

Hi guyzz,

just a short message to inform you all I made a GUI to simplify the wireless configuration and data gathering.
It supports real-time plots and much more!

https://github.com/VirtuContraFurore/jSmartPower3



Every feedback would be extremely appreciated <3

Regolo

[mad_ady]

Nice! Hope to give it a try next week!

[mctom]

Great!
I think the output format of SmartPower3 has changed at some point. Have you tested it with the most recent firmware?

Oh I think you did, if it works via wifi.

[paulcrawford]

Thanks for this logging tool. I did have one issue to get it running in MacOS 11.6.5:

Code: Select all

java -jar jSmartPower3/release/jSmartPower-1.0.jar

did not work. Instead I had to run:

Code: Select all

cd jSmartPower3/release
java -jar ./jSmartPower3-1.0.jar

Everything worked perfectly after that.

Two additional controls would improve the tool:

1. Ability to change the line colours for the three variables.
2. Ability to change the y-axis max and min scale values to allow the removal of some of the data noise.

Very valuable tool and excellent implication.

[paulcrawford]

Another minor issue: if the Voltage checkbox in Plot is unchecked the y-axis label for Voltage does not disappear but the scale values are repeated from Power if its checkbox is checked. Similarly with Current only selected Voltage still shows but with the Current scale values.

Can you explain a little bit more about how the "Time scale" value affects the plotted values? Two graphs are attached, one with a Time scale of minutes and one with seconds. The periodic spikes that show in "seconds" disappear in "minutes".

minutes graph.pdf

(143.86 KiB) Downloaded 7 times

seconds graph.pdf

(153.48 KiB) Downloaded 8 times

[Regolo]

Hi Paul, an enourmous thank you for having tried out my tiny project!

Okay, so:
1. changing colours could be implemented - why not - any hint about where to put another button in the GUI?
2. changing min and max value of y axis is painful because that would involve setting indpendent boundaries for current, voltage and power... I really don't have any clue on how to do this easily and user friendly
3. maybe the graph library is not getting that I removed the voltage series, could you provide a screenshot?
4. well posed question, thank you. Time scale skips data points - it does not apply any filtering! - that would cause the glitches you observe. The purpose of time scale is to reduce the burden of plotting lots and lots of points. But of course, just skipping data (eg plotting one point every ten) distorts heavily the high frequency components of the signal.

I believe that the solution could be the following: before picking a point every X to plot, apply a low pass filter to the data series (something like an average moving filter would do the job) and then plot the filtered data. The problem is that the would require to repeat the filtering every time a new point is added. But I introduced time scale to reduce data plotting burden and apply a filter would point in the opposite direction, and that's why I haven't implemented it so far. But maybe it's worth the effort... I dunno! What's your opinion?

[mctom]

I have no idea abot Java and what can be done or not, but here goes my 3 grosze, based on rather usable interface of LTspice:

1. I'd expect color settings after right-clicking plot legend (by the way the default colors should be green and blue. It's a common practice because it addresses the most common type of color blindness across population)
2. Same with axis range, just right-clicking on it and set min-max (and a checkbox for automatic, as an alternative to manual)

4. That's an excellent idea to discard points from the plot to save time on rendering - LTSpice doesn't do that and sometimes it literally redraws a plot for 1 minute straight. Drives me mad every time.
Here's what you could do:
- group data points into "buckets", so you end up with as many buckets as horizontal pixels on the graph
- In each bucket, identify minimum and maximum values and draw a vertical line in lighter color joining them
- On top of that, draw a point in solid color which is a median value of the bucket.

This way the user will be presented with the most meaningful information: Current peaks and general trend.
That's why I would advise against smoothing data in any way - current peaks are important and always should be visible.

EDIT:
Another way to represent reduced data is to draw three plots the same way it is already done:
- Minimum from each bucket, in lighter color
- Maximum, similarly
- Median in solid color
That of course won't make sense until buckets are bigger than 3.

[paulcrawford]

1. changing colours could be implemented - why not - any hint about where to put another button in the GUI?

I would put the button below the legend since it has the present colours.

2. changing min and max value of y axis is painful because that would involve setting indpendent boundaries for current, voltage and power... I really don't have any clue on how to do this easily and user friendly

The reason I ask about this is that most graphing software seems to look at the data to determine the max and min values. This means that noise is accentuated since there is almost never a value of "0" in the data. So in the screenshot below the power data seems to spike wildly but if the y-axis were down to "0" the real percentage changes would appear to be much less. In some cases the user may want to just see the max and min of the data so it definitely should be an option only to show the min value as "0".

3. maybe the graph library is not getting that I removed the voltage series, could you provide a screenshot?

screenshot power only.pdf

(223.05 KiB) Downloaded 5 times

As you can see the Voltage Plot is not checked but y-axis label shows up anyway.

4. well posed question, thank you. Time scale skips data points - it does not apply any filtering! - that would cause the glitches you observe. The purpose of time scale is to reduce the burden of plotting lots and lots of points. But of course, just skipping data (eg plotting one point every ten) distorts heavily the high frequency components of the signal.

I believe that the solution could be the following: before picking a point every X to plot, apply a low pass filter to the data series (something like an average moving filter would do the job) and then plot the filtered data. The problem is that the would require to repeat the filtering every time a new point is added. But I introduced time scale to reduce data plotting burden and apply a filter would point in the opposite direction, and that's why I haven't implemented it so far. But maybe it's worth the effort... I dunno! What's your opinion?

I have developed a program in "C" that runs on an Odroid n2+ with Ubuntu 20.04 server. I collects data from many different sensors and puts it into a data structure that contains the present and past value of each variable that is collected. Also within the data structure a delta value is stored for each variable. As each value for a specific time is collected and recored in the present value part the data structure it is compared with the past value and only if at least one variable has exceeded the delta value does the entire set of present values is written to a MySQL database for persistent storage. Then a flag is set that triggers the plotting software to read the database to obtain all of the data available for the extent of time x-axis required. If there is no need for persistent storage you can just have another data structure that contains only the data in which at least one delta value has been exceeded and use that data for plotting. While a low pass filter will minimize the data that needs to be plotted, it will miss any spikes that occur while checking for the delta exceedance will show every spike while still avoiding the overhead of plotting virtually identical data, the changes in which does not exceed the delta value.This delta value for each variable does need to be able to be edited in real time as it is often difficult to know the best value at the outset.

I do not know how easy or hard the above approach would be in Java, but it may be worth exploring. I am tracking 51 variables but you only have three.

[paulcrawford]

@Regolo, I have found a few small issues with the wifi panel. Everything else is working fine.

As you can see in the screenshot below, wifi is noted to be not connected although the data continues to arrive. SSID: text box does not respond to clicking. UDP addr: does respond but the edited value from the dialog box does not go into the text box. Clicking on "Refresh device wifi status" does nothing. All of this is a little misleading. Does the serial connection need to be active to get and update this information?

screen shot wifi.pdf

(316.65 KiB) Downloaded 3 times

[paulcrawford]

@Regolo, I was just clicking on the "Main" menu and "About" when a long series of errors occurred. There is more but this is the beginning:

Code: Select all

Exception in thread "AWT-EventQueue-0" java.lang.ArrayIndexOutOfBoundsException: 38771
	at org.knowm.xchart.internal.Utils.getDoubleArrayFromNumberList(Utils.java:110)
	at org.knowm.xchart.XYChart.updateXYSeries(XYChart.java:331)
	at it.sssup.jsmartpower3.ChannelPlotPanel$ChannelPlot.repaintChart(ChannelPlotPanel.java:398)
	at it.sssup.jsmartpower3.DataLogger$3.run(DataLogger.java:451)
	at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:728)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
Exception in thread "AWT-EventQueue-0" java.util.ConcurrentModificationException
	at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:911)
	at java.util.ArrayList$Itr.next(ArrayList.java:861)
	at org.knowm.xchart.internal.Utils.getDoubleArrayFromDateList(Utils.java:124)
	at org.knowm.xchart.XYChart.updateXYSeries(XYChart.java:330)
	at it.sssup.jsmartpower3.ChannelPlotPanel$ChannelPlot.repaintChart(ChannelPlotPanel.java:398)
	at it.sssup.jsmartpower3.DataLogger$3.run(DataLogger.java:451)
	at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:728)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
Exception in thread "AWT-EventQueue-0" java.util.ConcurrentModificationException
	at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:911)
	at java.util.ArrayList$Itr.next(ArrayList.java:861)
	at org.knowm.xchart.internal.Utils.getDoubleArrayFromDateList(Utils.java:124)
	at org.knowm.xchart.XYChart.updateXYSeries(XYChart.java:330)
	at it.sssup.jsmartpower3.ChannelPlotPanel$ChannelPlot.repaintChart(ChannelPlotPanel.java:398)
	at it.sssup.jsmartpower3.DataLogger$3.run(DataLogger.java:451)
	at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:728)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
Exception in thread "AWT-EventQueue-0" java.util.ConcurrentModificationException
	at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:911)
	at java.util.ArrayList$Itr.next(ArrayList.java:861)
	at org.knowm.xchart.internal.Utils.getDoubleArrayFromDateList(Utils.java:124)
	at org.knowm.xchart.XYChart.updateXYSeries(XYChart.java:330)
	at it.sssup.jsmartpower3.ChannelPlotPanel$ChannelPlot.repaintChart(ChannelPlotPanel.java:398)
	at it.sssup.jsmartpower3.DataLogger$3.run(DataLogger.java:451)
	at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:728)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
Exception in thread "AWT-EventQueue-0" java.lang.IllegalArgumentException: X and Y-Axis sizes are not the same!!!
	at org.knowm.xchart.internal.series.AxesChartSeriesNumericalNoErrorBars.replaceData(AxesChartSeriesNumericalNoErrorBars.java:63)
	at org.knowm.xchart.XYChart.updateXYSeries(XYChart.java:365)
	at org.knowm.xchart.XYChart.updateXYSeries(XYChart.java:328)
	at it.sssup.jsmartpower3.ChannelPlotPanel$ChannelPlot.repaintChart(ChannelPlotPanel.java:398)
	at it.sssup.jsmartpower3.DataLogger$3.run(DataLogger.java:451)
	at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:728)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
Exception in thread "AWT-EventQueue-0" java.util.ConcurrentModificationException
	at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:911)
	at java.util.ArrayList$Itr.next(ArrayList.java:861)
	at org.knowm.xchart.XYChart.getDataType(XYChart.java:258)
	at org.knowm.xchart.XYChart.updateXYSeries(XYChart.java:325)
	at it.sssup.jsmartpower3.ChannelPlotPanel$ChannelPlot.repaintChart(ChannelPlotPanel.java:398)
	at it.sssup.jsmartpower3.DataLogger$3.run(DataLogger.java:451)
	at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:728)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

If you need more please advise.

Thanks

[Regolo]

Hi Paul, sorry for the delay. Thank you for exploring and testing my piece of software!!! I'm quite busy during these weeks for preparing university exams. However, let me explain what you are seeing:

1) Let's start by the last exception. It seems something went wrong with Java threads writing things when they shouldn't. I have enforced synchronization using mutexes and this shouldn't happen. Data is constantly polled, hence maybe some graphic thread woke up at the wrong time and complained that someone else touched its data. But I need to reproduce that issue to be sure of the cause.

2) This is my fault! I should have made that more clear... The wifi status is "updated" if and only if you click the button "refresh wifi status". I guess that you have already configured wifi and after application relaunch you would receive data, but wifi status is not consistent.

3) This is a poor design of the smart power 3's firmware. To enter wifi config mode you need to send CTRL+C and to exit same. If for some reason something goes wrong with the software communicating with the smart power has now way to reset the status to a known state. Let's suppose my code does a mess at a certain point leaving the device in a unkown configuration. There's no sequence to restor it to a known state, since sending CTRL+C just flip the current (unknown) state. Firmware should be patched in that sense and provide a hot key combination to exit o enter wifi status from any state (already in or out).

[Regolo]

Nonetheless, sometimes wifi config buttons DO take while since the smart power replies very slowly, so wait patiently a few seconds. This is especially true when you want to scan wifi access points, you have to wait even 1 or 2 minutes!

[Regolo]

One last consideration: in order to do wifi config stuff, serial port must be connected! If think I have set up my code to simply do nothing if you press wifi config buttons while serial port is not connected!