|
TN200302B
- Internet Telemetry
Vaughan Wesson, February
2003.
Introduction
This technical note covers
the general philosophy of using the Internet to transfer data, the various components
in the system and then describes in detail how this is implemented with the
Kelunji D Series recorder.
A useful glossary
of Internet terms is available as TN200302D.
Internet telemetry is the
term used by the SRC to describe the use of the Internet (or more precisely
IP based communications) as a significant link in the chain between a recorder
and the analysis centre. There are a number of advantages of this system including:
- The cost of sending the
data is independent of the distance travelled
- The Internet is a very
reliable communications medium with much built in redundancy
- TCP/IP is a well tested,
reliable set of communications protocols providing the basis for a number
of higher level protocols such as FTP (file transfers), SMTP/POP3 (email)
and HTTP (Web)
- Multiple communication
sessions can be handled simultaneously on one IP based link. For example an
FTP file transfer and a Web exchange can be handled simultaneously without
any additional protocols
- Access points to the
Internet are becoming more common and access is becoming cheaper as more and
more companies and individuals use the Internet for business and pleasure
Components of the System
The components of the system
can generally be described as the chain:
Kelunji <-> ISP <-> Internet <-> ISP <-> Analysis Centre
That is, the Kelunji connects
to an Internet Service Provider (ISP) which is connected to the Internet. Similarly,
the analysis centre (or data management centre or whatever) is also connected
via an ISP to the Internet. The IP based protocols (see glossary)
are used to transfer (route) the information from the Kelunji, through its ISP,
through the Internet, through the other ISP to the analysis centre and vice
versa.
The Internet
The Internet is often drawn
as a fussy cloud or some other vague shape as it is difficult to describe it
as having any physical form. It is a collection of cables and satellite links
providing a series of communication channels between devices. It is the Internet
Protocol (IP) that is the common factor allowing all the various devices to
communicate with each other.
Internet Service Providers
The ISP's provide the ability
for normal users to connect to The Internet. They will typically have more than
one communication channel to The Internet backbone to provide a secure service
and allow users to connect to them using various means. The most common means
are:
- Dial up modem
- ADSL modem
- Cable modem
- ISDN
- Radio modem
- High speed digital cable
For example, ES&S has
an ADSL connection to its ISP, while at home I have a cable connection to mine
and many of our seismographs use a dial up modem connection. The main differences
between the systems are speed, reliability and cost.
A useful resource for obtaining
details about local ISP's in Australia can be found here.
With this site you can search for an ISP using either a phone number or geographically.
If just sending triggered
data files, a cheap Internet account may be used. Either one based on connection
time (around $1 per hour) or a fixed monthly account with a limited number of
hours can be used. However, for continuous data a more expensive account allowing
for continuous connection is required. Finally, a fixed IP address for the site
will usually cost more again.
Analysis Centre
The analysis centre is where
all the data ends up. For the Kelunji Internet telemetry services, all that
is required in (or associated with) the analysis centre is an FTP server. This
is what the Kelunji will send the data files to. The Kelunji has settings for
a user name and password for connecting to the FTP server and a directory may
also be specified for where the files are to be stored. As of version V5.11A,
the Kelunji supports both active and passive mode FTP file transfers, so should
work with all FTP servers.
To make best use of the
data, the SRC's eqLogger and eqWatch programs are designed to process continuous
and triggered files respectively as produced by the Kelunji.
Kelunji Data Communications
via the Internet
The Kelunji contains software
to perform IP based communications and can therefore connect directly to an
ISP or a computer. At present, Guria (V5.10A through to V5.11A) provides the
following Internet based services:
- Sending of triggered
data files via FTP. These may contain none, some or all of the channels being
sampled and may optionally be compressed. If no channels are selected, only
the trigger time and amplitude information are sent.
- Sending of continuous
data files via FTP. These may contain some or all of the channels being continuously
recorded and may optionally be compressed.
- Sending of a State-of-Health
file via FTP at a specified interval. Since every trigger file also includes
this state of health information, if a trigger file has been sent in the required
time interval then the extra file is not sent.
- Running an FTP server.
This allows standard PC's to examine the files present, retrieve them, delete
them and so on as required.
- Running a Web server.
This allows a user to connect using a standard Web browser. At present many,
but not all, functions of the Kelunji are available through the Web based
interface. For example, the recorders operation can be checked, data files
viewed and some settings altered.
In the near future, we will
be adding email support. This will provide some or all of the following additional
services:
- Sending of triggered
data files via email
- Sending state of health
files via email
- AutoDRM for a variety
of information including waveforms
- Receiving email commands
to change operating parameters
The listed services are
available through either an Ethernet connection or a serial connection using
PPP. The Ethernet interface is faster but uses more power and may not be available
in some situations. The PPP interface will probably be the most common in the
field.
Network Interfaces
The Kelunji Internet connectivity
is provided through what it calls its "Network interfaces". These
are set up in the "Settings:Serial:Network interfaces" menu. For versions
up to at least V5.11A only one interface is supported, but the code is written
to support up to four separate interfaces - for example, one Ethernet and three
PPP. Each interface may be operated in one of three modes:
- Mode 0 - Permanent
- In this mode, the Kelunji
will try and keep the interface open at all times. For example if a phone
call through a modem is dropped for some reason, the software will recognise
this after some time and attempt to re-establish the connection.
- Mode 1 - On Demand
- In this mode, the Kelunji
will normally have the modem set waiting to answer a phone call and establish
a PPP connection. However, when it is required to send a file (e.g. after
triggering), it will initiate a connection to the ISP as specified. After
the file has been transferred, the connection will be closed and the Kelunji
will revert to waiting for a phone call.
- Mode 2 - Send Only
- In this mode, the connection
is usually closed. When it is required to send a file, the connection is opened,
the file sent and then the connection is closed again.
Obviously, the mode to use
will depend on what services are being used. Mode 0 (permanent) is the usual
mode if continuous data files are being sent or either the FTP or Web server
is required. Mode 2 (send only) will yield the lowest cost and power consumption
if using a modem powered from the Kelunji.
Each interface has its own
IP address. This may be obtained as part of the PPP negotiation process. It
also has its own network mask, gateway address and up to three DNS server addresses.
The PPP negotiation will also attempt to obtain a DNS server address from the
ISP. The DNS servers are queried in turn to convert a domain name to an IP address
if required.
PPP Interfaces
The Kelunji contains up
to four serial ports that may be used for PPP connections. These are the Console
("Serial") port and the three Digitiser ports. Where possible, it
is best to leave the Console port for a user terminal and use one of the Digitiser
ports for the modem to connect to the ISP.
For PPP interfaces additional
settings are available.
- An initialisation string
for the modem. A typical string would be AT&F&D0&K0L3
- The power for the modem
may come from the Digitiser port, the alarm/timing port or neither. Where
possible it is best to power the modem from the Kelunji as then the Kelunji
can power cycle the modem if required
- A number of seconds delay
may be specified after the modem power is applied. For a landline modem this
may be 5 or 10 seconds, but for a GSM modem it may be 20 or 30 seconds to
allow the modem to "find" the network
- The phone number of the
ISP may be specified. Alternatively, if a direct cable to a PC is desired
then the strings WINNT or WIN95 may be specified to indicate this. The machine
being connected to must be running Remote Access Server or a similar program
- Limits on the number
of phone calls per day and minimum number of minutes between calls are possible.
These limits are discussed below
- The user name and password
to supply to the ISP
The limits on the phone
calls are required to cope with situations where the Kelunji cannot establish
a connection to the ISP for some reason. The first limit is the number of calls
allowed in one 24 hour period from 0000 UT one day to 0000 UT the next day.
Each time the Kelunji attempts to dial the ISP is considered one phone call
(note that it does not have to actually connect for it to be counted). The second
parameter forces a minimum time between phone calls. This is designed to cope
with periods of congestion on the GSM network for example. The time is measured
from when the previous call was attempted.
Ethernet Interface
A single 10 Mbit/s Ethernet
interface is provided on the Kelunji recorder. This can be used for either twisted
pair or coaxial cables. With the Ethernet enabled, the Kelunji power consumption
increases considerably. This is from a combination of the power drawn by the
Ethernet transceiver itself and the fact that the processor is required to run
at high speed all the time rather than sleeping when no tasks are to be performed.
When using the Ethernet
interface, the network mask and gateway address will usually be required.
An Ethernet port has a unique
address assigned to it by the manufacturer, known as its MAC address. This is
a 48 bit number usually written as six 8 bit numbers each written in hex and
separated by either a hyphen or a colon. For example 08:00:69:02:1D:FC or 08-00-69-02-1D-FC.
Each manufacturer obtains an allocation of address from the Institute of Electrical
and Electronics Engineers, Inc (IEEE) usually by obtaining an "Organisationally
Unique Identifier" (OUI) which is a three byte value that may be used as
the first three bytes of an Ethernet address. Environmental Systems and Services
has the OUI of 00-0B-22. For the Kelunji D series, its Ethernet MAC address
is obtained by combing our OUI with the serial number of the DK board. Therefore
the MAC address of DK board number 77 would be 00-0B-22-00-00-4D.
File Transfer Steps
This section lists the steps
the Kelunji takes in sending a file via FTP. It includes the messages that may
appear if the diagnostic debug messages are enabled.
There is a task in the recorders
software that is dedicated to sending files via FTP. This task may be "triggered"
by any of the following events:
- Expiration of a dedicated
timer (180 seconds pre V5.11A, now 133 seconds)
- Creation of a trigger
file or continuous recording file to be sent via FTP
- Creating of a state-of-health
file
- Creation of an Intensity
peak file
- Creation of a Finish
Time file
- A Network Interface (PPP
or Ethernet) successfully opening
Any files to be sent via
FTP are written to either the FTP_CONT or FTP_TRIG directories. Continuous data
files are stored in the FTP_CONT directory, all others are in the FTP_TRIG directory.
Note that when using multiple drives, these directories will be present on ALL
drives.
However the task is triggered,
the same steps are followed. These are:
- If triggering is enabled
and we have selected that trigger files will be FTPed, or we are calculating
intensity and we have selected to FTP Intensity peaks THEN check and send
files from the FTP_TRIG directory
- If continuous recording
is enabled and we have selected that continuous data files will be FTPed THEN
check and send files from the FTP_CONT directory
- Go back to sleep waiting
for the next "trigger"
Check and Send Files From
a Directory
| Step |
Possible
Debug Messages |
Action
taken on Failure |
| For each drive in turn,
count the number of files in the directory. A maximum of 50 files will be
processed at a time. These are not necessarily the earliest 50, they are
just processed in the order they appear in the directory system |
|
|
| If some files are found
on this drive, send them (see below) |
"Only sent xx
of the yy files successfully" if not all files were sent |
Ignored |
Send a Group of Files
| Step |
Possible
Debug Messages |
Action
Taken on Failure |
| Sort the files by file
name. This also means that they are sorted chronologically. |
|
|
| Check whether the interface
is being used by a dial in call. |
"Interface already
in use" |
Transfer aborted |
| Check whether we are
allowed to make a phone call. If the interface is not using a modem, this
will always be true. Similarly, if the interface is already open we are
OK. Otherwise, the time since the last call is checked and then the number
of calls this day. |
"Not enough time
elapsed for i/f #x" or "Too many calls this day for i/f #x" |
Transfer aborted |
| Attempt to open the
Interface (see below) |
If the open succeeded,
"Opened interface #x through Ethernet port" or "Opened interface
#x to phonenumber, username, password". If the open failed, "Could
not open interface #x to phonenumber, username, password" |
Transfer aborted |
| For PPP links, check
that the connection is still "up" using an LCP echo request |
|
Transfer aborted |
| Lookup the IP address
specified for the FTP server |
"Resolved address
domain.name to nnn.nn.nn.nn" or "Could not resolve address domain.name" |
Transfer aborted |
| Attempt to connect
to the FTP server |
"Connect failed,
status=nn". The common status values are -3 for a bad password, -4
for a bad user name or -1 for a general connection problem. |
If the interface is
in Mode 0 (Permanent) and is a PPP interface, then the connection is closed
as we may well have lost our connection. File transfers are abandoned. |
| Set the FTP file transfer
mode to image (binary) |
"Problem setting
type to image, status=n" |
Quit the FTP server
and abort transfers |
| If a directory is specified,
ask the FTP server to change to the indicated directory |
"Problem setting
the directory to xxxx, status=n". A status value of -1 indicates that
the directory does not exist or the user does not have permission |
Quit the FTP server
and abort transfers |
| Send each file in turn
(see below). Delete the file once it has been successfully sent. |
"Sending filename" |
Ignored. Move on to
next file |
| Close the FTP session |
|
|
| If the interface is
not to be left open, then we now close it (see below) |
|
|
Open an Interface
| Step |
Possible
Debug Messages |
Action
Taken on Failure |
| If it is already open
then we are done |
|
|
| If it was opening waiting
for a phone call then we must close it first |
|
|
| If the interface is
already in the process of opening (having being triggered already) then
we abandon trying to open it again |
|
|
| Open an Ethernet interface
or a PPP interface as required (see below) |
|
|
| Wait until the interface
is open. For a PPP interface, this may take some time as this includes the
modem dialling and training with its remote end, logging in to the ISP and
the PPP handshaking. |
|
Hang up the modem and
indicate failure |
| If the interface opened
OK, we enter any DNS servers associated with that interface to the active
DNS server table and we are done |
|
|
Open an Ethernet Interface
| Step |
Possible
Debug Messages |
Action
Taken on Failure |
| Enable the Ethernet
transceiver and set up the Ethernet port |
|
Give up |
| Set the IP address
and network mask for the port |
|
|
| Notify the Web server
of the new IP address |
|
|
| Add the gateway address
to the routing table |
|
|
Open a PPP Interface
| Step |
Possible
Debug Messages |
Action
Taken on Failure |
| Check that we have
access to the indicated serial port |
|
Give up |
| Check that PPP links
have not been disabled (using the Disarm button) |
|
Give up |
| Check whether the interface
is already opening |
|
Give up |
| Spawn a separate task
to perform the connection |
|
Give up |
| Open the serial port
at the desired sample rate |
|
Return failure indication |
| Set the IP address
and network mask for the port |
|
Return failure indication |
| Set up the PPP negotiation
constraints |
|
|
| Switch on the modem
power if required and wait the indicated delay |
|
|
| Dial the modem and
attempt to connect to the ISP (see below) |
Many. See table below |
Hangup the modem, switch
off modem power and return failure indication |
| Perform the PPP negotiation |
A number are possible
from the RTIP software |
Hangup the modem, switch
off modem power and return failure indication |
| Add the remote address
as a default gateway in the routing table |
|
Hangup the modem, switch
off modem power and return failure indication |
| Notify the Web server
of the new IP address |
|
|
Dial Modem and Connect
to ISP
| Step |
Possible
Debug Messages |
Action
Taken on Failure |
| Update details about
number of calls and time of last call |
|
|
| Send init string to
modem |
The various modem
exchanges and "Dialling phonenumber failed" |
Hang up and return
failure |
| Send dial string to
modem |
The various modem exchanges |
Hang up and return
failure |
| On connection, note
the call start time |
|
|
Send a File
| Step |
Possible
Debug Messages |
Action
Taken on Failure |
| Open the file |
"Problem opening
the file to send, status=n". |
Return failure |
| Transfer the file giving
it a temporary name on the destination machine |
FTP commands and answers.
"Problem sending the file, status=n". |
Return failure |
| Rename the file on
the destination machine |
FTP commands and answers.
"Rename from failed. status=n", "Rename to failed. Status=n". |
Return failure |
Close the Interface
| Step |
Possible
Debug Messages |
Action
Taken on Failure |
| If it is already closed
then we are done |
|
|
| If it is an Ethernet
interface, disable the transceiver and close down the Ethernet controller |
|
|
| If it is a PPP interface,
signal any open connections to close then wait for the PPP link to shut
down. Hang up the modem, close the serial port and switch off power to the
modem |
|
|
| Remove any DNS servers
from the active list that were for this interface |
|
|
|