This technical note covers the general philosophy of using the Internet to transfer data, the various components in the system and then describes in detail how this is implemented with the Kelunji D Series recorder.
A useful glossary of Internet terms is available as TN200302D.
Internet telemetry is the term used by the SRC to describe the use of the Internet (or more precisely IP based communications) as a significant link in the chain between a recorder and the analysis centre. There are a number of advantages of this system including:
The components of the system can generally be described as the chain:
Kelunji <-> ISP <-> Internet <-> ISP <-> Analysis Centre
That is, the Kelunji connects to an Internet Service Provider (ISP) which is connected to the Internet. Similarly, the analysis centre (or data management centre or whatever) is also connected via an ISP to the Internet. The IP based protocols (see glossary) are used to transfer (route) the information from the Kelunji, through its ISP, through the Internet, through the other ISP to the analysis centre and vice versa.
The Internet is often drawn as a fussy cloud or some other vague shape as it is difficult to describe it as having any physical form. It is a collection of cables and satellite links providing a series of communication channels between devices. It is the Internet Protocol (IP) that is the common factor allowing all the various devices to communicate with each other.
The ISP's provide the ability for normal users to connect to The Internet. They will typically have more than one communication channel to The Internet backbone to provide a secure service and allow users to connect to them using various means. The most common means are:
For example, ES&S has an ADSL connection to its ISP, while at home I have a cable connection to mine and many of our seismographs use a dial up modem connection. The main differences between the systems are speed, reliability and cost.
A useful resource for obtaining details about local ISP's in Australia can be found here. With this site you can search for an ISP using either a phone number or geographically.
If just sending triggered data files, a cheap Internet account may be used. Either one based on connection time (around $1 per hour) or a fixed monthly account with a limited number of hours can be used. However, for continuous data a more expensive account allowing for continuous connection is required. Finally, a fixed IP address for the site will usually cost more again.
The analysis centre is where all the data ends up. For the Kelunji Internet telemetry services, all that is required in (or associated with) the analysis centre is an FTP server. This is what the Kelunji will send the data files to. The Kelunji has settings for a user name and password for connecting to the FTP server and a directory may also be specified for where the files are to be stored. As of version V5.11A, the Kelunji supports both active and passive mode FTP file transfers, so should work with all FTP servers.
To make best use of the data, the SRC's eqLogger and eqWatch programs are designed to process continuous and triggered files respectively as produced by the Kelunji.
The Kelunji contains software to perform IP based communications and can therefore connect directly to an ISP or a computer. At present, Guria (V5.10A through to V5.11A) provides the following Internet based services:
In the near future, we will be adding email support. This will provide some or all of the following additional services:
The listed services are available through either an Ethernet connection or a serial connection using PPP. The Ethernet interface is faster but uses more power and may not be available in some situations. The PPP interface will probably be the most common in the field.
The Kelunji Internet connectivity is provided through what it calls its "Network interfaces". These are set up in the "Settings:Serial:Network interfaces" menu. For versions up to at least V5.11A only one interface is supported, but the code is written to support up to four separate interfaces - for example, one Ethernet and three PPP. Each interface may be operated in one of three modes:
Obviously, the mode to use will depend on what services are being used. Mode 0 (permanent) is the usual mode if continuous data files are being sent or either the FTP or Web server is required. Mode 2 (send only) will yield the lowest cost and power consumption if using a modem powered from the Kelunji.
Each interface has its own IP address. This may be obtained as part of the PPP negotiation process. It also has its own network mask, gateway address and up to three DNS server addresses. The PPP negotiation will also attempt to obtain a DNS server address from the ISP. The DNS servers are queried in turn to convert a domain name to an IP address if required.
The Kelunji contains up to four serial ports that may be used for PPP connections. These are the Console ("Serial") port and the three Digitiser ports. Where possible, it is best to leave the Console port for a user terminal and use one of the Digitiser ports for the modem to connect to the ISP.
For PPP interfaces additional settings are available.
The limits on the phone calls are required to cope with situations where the Kelunji cannot establish a connection to the ISP for some reason. The first limit is the number of calls allowed in one 24 hour period from 0000 UT one day to 0000 UT the next day. Each time the Kelunji attempts to dial the ISP is considered one phone call (note that it does not have to actually connect for it to be counted). The second parameter forces a minimum time between phone calls. This is designed to cope with periods of congestion on the GSM network for example. The time is measured from when the previous call was attempted.
A single 10 Mbit/s Ethernet interface is provided on the Kelunji recorder. This can be used for either twisted pair or coaxial cables. With the Ethernet enabled, the Kelunji power consumption increases considerably. This is from a combination of the power drawn by the Ethernet transceiver itself and the fact that the processor is required to run at high speed all the time rather than sleeping when no tasks are to be performed.
When using the Ethernet interface, the network mask and gateway address will usually be required.
An Ethernet port has a unique address assigned to it by the manufacturer, known as its MAC address. This is a 48 bit number usually written as six 8 bit numbers each written in hex and separated by either a hyphen or a colon. For example 08:00:69:02:1D:FC or 08-00-69-02-1D-FC. Each manufacturer obtains an allocation of address from the Institute of Electrical and Electronics Engineers, Inc (IEEE) usually by obtaining an "Organisationally Unique Identifier" (OUI) which is a three byte value that may be used as the first three bytes of an Ethernet address. Environmental Systems and Services has the OUI of 00-0B-22. For the Kelunji D series, its Ethernet MAC address is obtained by combing our OUI with the serial number of the DK board. Therefore the MAC address of DK board number 77 would be 00-0B-22-00-00-4D.
This section lists the steps the Kelunji takes in sending a file via FTP. It includes the messages that may appear if the diagnostic debug messages are enabled.
There is a task in the recorders software that is dedicated to sending files via FTP. This task may be "triggered" by any of the following events:
Any files to be sent via FTP are written to either the FTP_CONT or FTP_TRIG directories. Continuous data files are stored in the FTP_CONT directory, all others are in the FTP_TRIG directory. Note that when using multiple drives, these directories will be present on ALL drives.
However the task is triggered, the same steps are followed. These are:
| Step | Possible Debug Messages | Action taken on Failure |
| For each drive in turn, count the number of files in the directory. A maximum of 50 files will be processed at a time. These are not necessarily the earliest 50, they are just processed in the order they appear in the directory system | ||
| If some files are found on this drive, send them (see below) | "Only sent xx of the yy files successfully" if not all files were sent | Ignored |
| Step | Possible Debug Messages | Action Taken on Failure |
| Sort the files by file name. This also means that they are sorted chronologically. | ||
| Check whether the interface is being used by a dial in call. | "Interface already in use" | Transfer aborted |
| Check whether we are allowed to make a phone call. If the interface is not using a modem, this will always be true. Similarly, if the interface is already open we are OK. Otherwise, the time since the last call is checked and then the number of calls this day. | "Not enough time elapsed for i/f #x" or "Too many calls this day for i/f #x" | Transfer aborted |
| Attempt to open the Interface (see below) | If the open succeeded, "Opened interface #x through Ethernet port" or "Opened interface #x to phonenumber, username, password". If the open failed, "Could not open interface #x to phonenumber, username, password" | Transfer aborted |
| For PPP links, check that the connection is still "up" using an LCP echo request | Transfer aborted | |
| Lookup the IP address specified for the FTP server | "Resolved address domain.name to nnn.nn.nn.nn" or "Could not resolve address domain.name" | Transfer aborted |
| Attempt to connect to the FTP server | "Connect failed, status=nn". The common status values are -3 for a bad password, -4 for a bad user name or -1 for a general connection problem. | If the interface is in Mode 0 (Permanent) and is a PPP interface, then the connection is closed as we may well have lost our connection. File transfers are abandoned. |
| Set the FTP file transfer mode to image (binary) | "Problem setting type to image, status=n" | Quit the FTP server and abort transfers |
| If a directory is specified, ask the FTP server to change to the indicated directory | "Problem setting the directory to xxxx, status=n". A status value of -1 indicates that the directory does not exist or the user does not have permission | Quit the FTP server and abort transfers |
| Send each file in turn (see below). Delete the file once it has been successfully sent. | "Sending filename" | Ignored. Move on to next file |
| Close the FTP session | ||
| If the interface is not to be left open, then we now close it (see below) |
| Step | Possible Debug Messages | Action Taken on Failure |
| If it is already open then we are done | ||
| If it was opening waiting for a phone call then we must close it first | ||
| If the interface is already in the process of opening (having being triggered already) then we abandon trying to open it again | ||
| Open an Ethernet interface or a PPP interface as required (see below) | ||
| Wait until the interface is open. For a PPP interface, this may take some time as this includes the modem dialling and training with its remote end, logging in to the ISP and the PPP handshaking. | Hang up the modem and indicate failure | |
| If the interface opened OK, we enter any DNS servers associated with that interface to the active DNS server table and we are done |
| Step | Possible Debug Messages | Action Taken on Failure |
| Enable the Ethernet transceiver and set up the Ethernet port | Give up | |
| Set the IP address and network mask for the port | ||
| Notify the Web server of the new IP address | ||
| Add the gateway address to the routing table |
| Step | Possible Debug Messages | Action Taken on Failure |
| Check that we have access to the indicated serial port | Give up | |
| Check that PPP links have not been disabled (using the Disarm button) | Give up | |
| Check whether the interface is already opening | Give up | |
| Spawn a separate task to perform the connection | Give up | |
| Open the serial port at the desired sample rate | Return failure indication | |
| Set the IP address and network mask for the port | Return failure indication | |
| Set up the PPP negotiation constraints | ||
| Switch on the modem power if required and wait the indicated delay | ||
| Dial the modem and attempt to connect to the ISP (see below) | Many. See table below | Hangup the modem, switch off modem power and return failure indication |
| Perform the PPP negotiation | A number are possible from the RTIP software | Hangup the modem, switch off modem power and return failure indication |
| Add the remote address as a default gateway in the routing table | Hangup the modem, switch off modem power and return failure indication | |
| Notify the Web server of the new IP address |
| Step | Possible Debug Messages | Action Taken on Failure |
| Update details about number of calls and time of last call | ||
| Send init string to modem | The various modem exchanges and "Dialling phonenumber failed" |
Hang up and return failure |
| Send dial string to modem | The various modem exchanges | Hang up and return failure |
| On connection, note the call start time |
Send a File
| Step | Possible Debug Messages | Action Taken on Failure |
| Open the file | "Problem opening the file to send, status=n". | Return failure |
| Transfer the file giving it a temporary name on the destination machine | FTP commands and answers. "Problem sending the file, status=n". | Return failure |
| Rename the file on the destination machine | FTP commands and answers. "Rename from failed. status=n", "Rename to failed. Status=n". | Return failure |
Close the Interface
| Step | Possible Debug Messages | Action Taken on Failure |
| If it is already closed then we are done | ||
| If it is an Ethernet interface, disable the transceiver and close down the Ethernet controller | ||
| If it is a PPP interface, signal any open connections to close then wait for the PPP link to shut down. Hang up the modem, close the serial port and switch off power to the modem | ||
| Remove any DNS servers from the active list that were for this interface |