Diagtool Help

Version	Release Date	Changes/Updates Description
3.0	07/13/16	Application uses new web responsive (Bootstrap) design Single-Station Plotting: Redesigned option page; Highcharts Interactive Javascript tool replaces the gnuplot tool for generating plots; Can display air gap data Station Diagnostics and Station Satellite Message web pages are merged into a single page with the tabs Station Satellite Message application now provides the GOES messages statistics of current day or your selected day Station Message Details web page: Completely redesigned; Now displays all GOES details at once. Note that due to the size of decoded GOES information, this page is not ideally viewed on mobile devices. Redesigned Diagtool Login System page Redesigned Diagtool Home page New Diagtool Help page Login Session now expires after 1 hour. User will be prompted to re-login after session is expired.
3.01	07/26/16	The DiagTool product has a capability to display the M1 battery voltage in the Station Configuration Details (Sensors) information and plots/data.<
3.1	08/24/16	The Login Session will expire after one idling hour or 24-hour. Air Gap QC Plotting Unit is updated for the web responsive and uses Highcharts tool for plotting. Single-Stations Plotting Unit can forward or backward number of days based on the selected start and end dates instead of by one day.
3.2	10/18/16	New 30-day GOES Messages Status Web Page: This page displays GOES messages status for the past 30 days in a mobile-friendly calendar view to be able to quickly view messages status for a given day. Clicking on a calendar entry will bring up message details for that day.
3.3	01/31/17	All Quality Tools web pages are updated for using the Bootstrap for the web responsive design.
3.4	08/01/17	Multi-Stations Plotting Unit is updated for the web responsive and uses Highcharts tool for interactive plots.
3.41	08/08/17	The Station Diagnostics GOES Messsage Decoder is now able to convert GOES messages for new conductivity sensor with '-7' flag with 3 bytes data and visibility & alarm code with '-O' flag.
3.42	12/12/17	The 30-Day GOES Messages Status Statistics web application is updated that displays the month calendar(s) much quicker. The script, calculateMessageStatistics.pl is restored and modified for generating daily GOES messages status statistics for all stations. The CGI application reads the previous day GOES messages status statistics files except today's.
3.43	01/24/18	The GOES Elevation information is added to the GOES DCP Transmission Configuration Details page.
3.4.4	01/30/18	The DiagTool Products have been updated for the Section 508 compliance. Most changes were made for the color contracts. The jQuery-UI tools package is updated to 1.12.1 from 1.11.4. The azimuth information is added to the GOES DCP Transmission Configuration Details page.
3.4.5	02/26/18	The GOES Signal Strength Time Series Plot is added in the DiagTool Single-Station Plotting app.
3.4.6	03/01/18	Updates the GOES Signal Strength Time Series Plot to use the unit label as decibels (dB).
3.4.7	03/07/18	The GOES Platform IDs Searchable List Page is added. The button of 'GOES Platform IDs List' is added in the main DiagTool page that will open the GOES Platform IDs list page.
3.5.0	03/21/18	MET QC Plotting Unit is updated for the web responsive and uses the Highcharts tool for interactive plots.
3.5.1	03/27/18	The time issues in the Single-Station Plotting's GOES signal strength plots for the Great Lakes stations are corrected.
3.6.0	05/22/18	The Current Sensor Low Battery & Transmit Voltage Monitoring System is added. The data is up to a week and the plots are only displayed only when the CU stations have low voltage/counts data.
3.6.1	06/21/18	Modified all HTML files for using updated federated-analytics.js with tag ID in the <script> code
3.6.2	07/12/18	Started to record the failed login attempts.
3.6.3	08/21/18	A new stored procedure was created to retrieve the data interval per sensors for the Single-Station Plotting Unit, Met QC Plotting Unit, and Multi-Stations Plotting Unit. The DataList.pm and Data_JSON.pm were updated for this reason.
3.6.4	08/24/18	A fix was performed in the DataList.pm due to sensor J1's data interval value zero that halted the Single-Station Plotting Unit.
3.6.5	10/10/18	The Diagtool Login System was updated to use the CO-OPS LDAP accounts, replacing the NOAA LDAP accounts. The NOAA e-mail address was no longer used for the Diagtool username. The NOAA employees who work in other line offices, they had to use CO-OPS Application Acces Management (AAMC) product to access the Diagtool web products.
3.6.6	12/12/18	The Diagtool Single-Station Plotting Unit displays dots/markers on all plots. This makes the hourly data points more visible with the 6-minutes data interval that prevents drawing lines between hourly data points. The dots/markers will appear on all time series plots, no matter if they are 6 minutes or hourly data.
3.6.7	05/02/19	The Diagtool Air Gap QC Plotting Unit was updated for using daylight saving time (DST) to adjust the local time data.
3.6.8	07/11/19	The GOES Signal Strength plotting option was added to the Multi-Stations Plotting Unit.
3.7.0	08/22/19	The Diagtool Single-Station Plotting Unit was updated for: Using latest version of Highcharts, v7.1.1. No longer using DataList.pm, using its own routines if not shared by other Diagtool apps and moved other subroutines to Diagtool.pm if shared with other Diagtool apps. Data Retrieval Time should be improved as it does not perform the date stamp check. Displays the selected sensor type data table to view.
3.7.1	08/27/19	Added the codes in the Single-Station Plotting Unit to handle the time data for the Great Lakes and River stations from local standard time to GMT time. All data ingestions for the Great Lakes and River stations use local standard time data as default instead of GMT.
3.7.2	10/22/19	Added two new DIS information displays: All Air Gap DIS Parameters and Station/DCP DIS Parameters Increased the precision numbers on the single-station plottings and air gap plottings.
3.7.3	11/04/19	Modifed the java command to be able to work with the non-NOAA account password that had a special character that caused the java CAAM app failed to recognize.
3.7.4	11/07/19	The DIS Current Prediction Station Information Page is added.
3.7.5	01/14/20	The Platform IDs List app uses the stored procedures to retrieve a list instead of the flat data file.
3.7.6	03/05/20	The Searchable Platform IDs List app includes the CU stations.
3.7.7	03/10/20	The time is added to the starting and ending dates for getting the DCP # of GOES SS.
3.7.8	03/24/20	The DIS-Current Prediction Station Information is updated to use correct the column number for the DB output. The stored procedure was updated on Jan. 2020 without the Diagtool developer's knowledge that the new column was added. This threw the PERL script that generates the list of stations, DCP and names off in the column number order.
3.7.9	04/30/20	Increased the OFS name character length limit from 5 to either 5 or 6 for the SHEF OFS Reports.
3.7.10	05/01/20	Modified the UNIX command from 'find...' to 'ls -tr' to get a listing of the Wallop data files that have the selected station ID. The 'find' command on the AWS system does not sort the listing that on the Station Satellite Message Diagnostics page, the time was not sorted correctly under the 'Received' page. The 'ls -tr' command works for the listing sort order.
3.7.11	05/05/20	Modified the javascript, diagtool_airgapplots.js, to add codes for handling the zero values of the outlier data that should be plotting. It only affects on the AWS systems.
3.7.12	07/28/20	Updated the Single-Station Plotting Unit to use Highcharts v8.1.2 that includes the offline-exporting module allowing for image export of plots without sending data to the external server. From the Highcharts document: Internet Explorer requires the canvg library in order to export to PNG and to export charts with embedded images. PDF export also requires the jsPDF and svg2pdf for all browsers. By default, the module will load these file from our server on demand, but the URL can be changed using the exporting.libURL option. If the scripts have already been loaded by the page, Highcharts will not load them again. Note that even when loading these files, no chart data is sent to our server. PDF does not support exporting with images and will fall back to the export server. With that module, the users should be able to download the PNG of the plots with at least 5 days range of data. The previous Highchart version was not able to generate the PNG with at least 4 days range of data.
3.8.0	08/06/20	Updated other Plotting Units to use Highcharts v8.1.2 that includes the offline-exporting module allowing for image export of plots without sending data to the external server.
3.8.1	09/09/20	Resolved the GOES Platform ID Searchable app issue by closing the stored procedure before calling other stored procedure.
3.8.2	09/15/20	The Satellite Message Details app under the Station Diagnostics was able to display the second messages of the same time properly.
3.8.3	09/24/20	The Station Diagnostics page with the two tabs was splitted into two single pages: Station Configuration Details and GOES/GTAG Message Diagnostic. This reduced a number of the database callings/connections as the Station Configuration Details page was not used frequency by the users.
3.8.4	09/29/20	Revised the stored procedure that returned values instead of the empty values due to the error. The SP expected to find the value from the ADDRESS or STATE_REF for the far-Pacific Islands stations like Midways Islands, Guam, etc.
3.9.0	04/30/21	Updated the Bootstrap from v3.2.3 to v4.6.0.
3.9.1	8/18/21	Updated the Single-Station Plotting Unit to use Data API for retrieving tide predictions, replacing the old Water Level Predictions Java application.
3.9.2	09/02/21	Single-Station Plotting Unit: Fixed the unit option for the DataAPI tide prediction.
3.9.3	09/06/21	Updated the Air-Gap Plotting Unit to use the Data API for retrieving tide prediction, replacing the SQL stored procedure.
3.9.4	09/15/21	Single-Station Plotting Unit: Added the codes to subtract the WL data and WL prediction with the metric option.
3.9.5	09/16/21	Single-Station Plotting Unit: Allowed 1-minute U1 data instead of 6-minutes marks and added 1-minute tide predictions to be used for the WL residuals of U1/Predictions.
3.9.6	11/16/21	Single-Station Plotting Unit: Added the Highchart's Boost module (boost.js) to speed up the time series plots interactives when the user adjust the Y-axis values. Due to the high number of data points, they slowed down the plots interactive performance.
3.9.7	12/02/21	All Diagtool web pages have the notification message capability. The notification message is only enabled under diagtool_alert.html to notify the Diagtool users any messages.
3.9.8	01/06/22	Improved the Multi-Stations Plotting Unit to correct the issues of the Great Lakes stations' time shift, wind gust spikes, and handle the Great Lakes stations' data time shift to GMT.
3.9.9	01/13/22	Rewrote the MET-QC Plotting Unit to eliminate one PM where the PM codes were not shared with other apps.
3.9.10	02/01/22	Fixed the codes for wind data in the Single-Station Plotting Unit.
3.9.11	03/29/22	In the Single-Station Plotting Unit, added a capability to disable/enable 'Selected Sensor Data' tab.
3.9.12	04/10/22	Under the Single-Station Plotting Unit, the Operational Forecast System (OFS) water level is displayed on the water level time series and the OFS WL - WL Obs time series plot is added.
3.9.13	04/18/22	Under the Single-Station Plotting Unit, any temperature (air, water, and air thermistor) data with -204.8/204.8 (from the database with a metric unit, will change to the missing value (-999.99).
3.9.14	05/09/22	Under the Single-Station Plotting Unit, any temperature (air, water, and air thermistor) data with -204.8/204.8 (from the database with a metric unit, will change to the missing value (-999.99).
3.9.15	06/01/22	For the CREX/SHEF Status Report page, the file extension is excluded from the passing paramater value due to the Web Application Firewall rule that disallows the file extension as a possible security vulnerability risk.
3.9.16	06/03/22	On the Single-Station Plotting Unit app, the second codes part for checking the verified water level data availability is disabled. Since the second codes part is fixed for invalid date entries in another DIAG ticket (unversion), this application gets two data set of verified water level.
3.9.17	07/11/22	The new sensor offset parameters for the Paros N1/T1 sensors are added on the Station DCP Metadata (DIS Parameters) web page.
3.9.18	08/19/22	Corrected the month and year header location in the GOES 30 Days Calendar web page.
3.9.19	07/25/23	Added a data column of the visibility alarm code on the Single-Station Plotting Unit's Data Display (Data tab) page and and added a column of the visibility alarm code in the CSV output.
3.9.20	09/12/23	Updated for handling the relative humidity (R1) and barometric pressure (F1) 'bad' data into a missing value for both Single-Station Plotting Unit and Multi-Stations Plotting Unit.
3.9.21	09/14/23	Removed the 'Lake_on_GMT' parameter (second line) on the Station/DCP Metadata - DIS Parameters web page.
3.10.0	12/18/23	Updated the Air Gap Plotting Unit for handling multi-Air Gap sensors (Q1/1, Q1/2, Q2/1 and Q2/2).
3.10.1	12/19/23	Updated the Single-Station Plotting Javascript to establish the water level color order so each WL sensor can have a same color for all WL-related time series plots.
3.10.2	01/02/24	Added a Log Out button on the main Diagtool Menu page. This will allow the users to log out that will end the users session.

Station Configuration Details and Satellite Message Description Help

Where can I view Satellite Message Details?
Where can I view Station Configuration page?
Where can I view the 30-day GOES Messages Status page?

Where can I view Satellite Message Details?: Click Station Diagnostics from the Diagtool main menu. Select a station and the application will retrieve today's GOES information by default. If interested in other day, change the Date field and click Retrieve. The application will then retrieve the GOES message for that day. Click the Received tab and select the GOES message to view detailed information.
Where can I view Station Configuration page?: Click Station Diagnostics from the Diagtool main menu. Select a station and after the application finishes retrieving today's GOES messages status information, click the configuration tab. The Station tab displays the DCP installation dates; the Installed tab lists all installed sensors and DCP components; the Sensors tab lists all sensor types configured and which sensors are presently disseminating; and the DCP tab lists satellite transmission parameters.
Where can I view the 30-day GOES Messages Status page?: Click Station Diagnostics from the Diagtool main menu. Select a station and when the application finishes retrieving today's GOES messages status information, click the Go to a '30-day GOES Message Diagnostics page' button to view the calendar day boxes with GOES message status information.

Single-Station Plotting Help

Click Single Station Plotting from the Diagtool main menu. Type all or part of the station name or station ID in the station box and click the station you are intersted in. Please note that the selected station name should be included station ID. Select the options and date range and click 'Get Plots'. This will bring up plots of all sensor configured and transmitting for the station.

How to disable and enable a plot line?
Why does no data show up?
How can I adjust the plot?
How do I view raw data?

How to disable and enable a plot line?

Click on the sensor ID/DCP # to enable or disable a line displaying on the plot in the sensors box.

Enable and Disable Plot Line Instruction

Why does no data show up?

There are many factors why the Diagtool product doesn't display data.
1) If the green processing bar with a message of 'Plots Generating, please wait...' is still displaying with at least one time series plot displaying or no plots display, the Javascript that generates the time series plot(s) might be crashed. Notify the Diagtool Support Team with your selected station and date entries.
2) Check the Data tab and see if your selected station has the data of the sensor(s) that you seek for. If not, see #3. If yes, notify the Diagtool Support Team including the missing plot description, station name, and date entries.
3) For the most common metadata issues that affect DIS, the users should check on DPAS/NPDB PowerBuilder for the following if the station/DCP is properly configured:

Is a platform ID configured for the station ID / DCP?
Is the specific sensor configured for the station ID / DCP?
Are sensor and datum offsets configured for the station ID / DCP?
If N1 / T1 or B1, are the appropriate gain/offsets configurated?

3a) If everything is configured in #3, go to the tab of "How to report an issue?".

How can I adjust the plot?

You can adjust the plot a few different ways. If you wish to change the Y-scale, you can type the new max and min values in the box to the right of the plot or use the up and down arrows to change these values and then click 'Update Y-axis'. If you wish to view data for the previous or next day, you can use the left and right arrows to the right of the plot. If you wish to zoom in to an area, you can click and drag the mouse cursor horizontally across the desired period. When you are done, click 'Reset Zoom' in the upper-right corner.

How do I view raw data?

If you hover the mouse cursor over the plot, you will be able to view the individual values for that time. This is an improvement over the previous Diagtool plots. A listing of raw sensor data for all sensors can be viewed by clicking on the Data tab in the upper-left. Most recent data will be towards the bottom. You can export raw data using the 'Export to CSV' button in the upper-left. You can also view a summary of sensor parameters such as min and max values, number of samples, mean and standard deviation to the right of each plot.

Multi-Stations Plotting Help

Click Mulit-Stations Plotting from the Diagtool main menu. Select a sensor first that will prompt a list of the stations where your selected sensor is in operational.

To select stations between station(s) you don't want to use, press and hold ctrl and click on the station names you want to use.
To select stations consecutive, press and hold shift and click on station names.

How to report an issue

Please email the Diagtool Support Team/ISD (see the footer/bottom of the Diagtool web page), please describe the issue you have faced, on which Diagtool product(s), station(s), and the date entries that you selected.