It
would appear that one of the main backbone Internet connections for
much of Utah - Utopia Networks - lost some key hardware in their
network core. The effect may have been worst in Northern Utah.
As
a result the ISP providing the service for WebSDR - which uses Utopia
as one of its main Internet providers - switched to a much lower-speed
backup to maintain some
connectivity for their customers - but this also caused extremely slow
connections to the WebSDR making it impossible to maintain an audio
connection at times with packet transit times often exceeding 3 seconds.
The degradation of service lasted from around 1900 to a bit before 2300 MDT. At about 0130 local time on 31 March the faulty hardware - which had been reset (with much crossing of fingers, no doubt) was replaced.
25 March, 2020: "High Boost" added to the Northern Utah WebSDR servers:
A button marked "High Boost" is now present on the Northern Utah WebSDR servers. When activated, this provides a 6 dB boost to audio with the curve of the response "knee" centered at 1500 Hz.
This button may be useful for high levels of DSP noise reduction which can have the effect of suppress higher audio frequencies.
The high frequency boost may also be beneficial for those with high-frequency hearing loss, to improve intelligibility overall.
19 March, 2020: Internet restrictions relaxed - for now.
Our Internet Service Provider has notified us that bandwidth restrictions imposed by theirInternet
backhaul providers have been lifted and with their knowledge, we are
rolling the
configurations of the WebSDR servers back to their original
configuration in terms of audio compression, waterfall speed, number of
users and time-out limits. These changes require the servers to
be reset so we are waiting for the number of users to drop before doing
so.
PLEASE REMEMBER
that Internet usage has increased dramatically in many areas owing to
the increased need by industry and education for remote access. Because of this you should expect heavy Internet usage that will likely affect connectivity to all web sites at times.
The
WebSDRs, being nearly real-time, can have only minimal buffering - unlike
many streaming services - which means that they are more affected by
slow-downs in Internet traffic as they cannot simply "pre-fetch" (buffer)
seconds/minutes of content to accommodate.
18 March, 2020: Utah earthquake
At 0709 MDT there was an earthquake (5.7 reported as of writing) that was centered just north of Magna, Utah - approximately 11 miles (17 km)
west of downtown Salt Lake City - just a "little one" for many
California transplants.
Reports
indicate that the most severe shaking was
very localized with power outages and damage to unreinforced masonry
structures in the immediate area and several trailer homes having
fallen off their piers. Elsewhere in the Salt Lake
valley the reports vary from "A loud truck going by" to "Holy $#!+".
Businesses and warehouses have reported loss of inventory due to
items having fallen off shelves and tipping/collapse of shelves.
As
of this writing, there are no reports of serious injuries. Dozens
of aftershocks have been felt all across the Salt Lake area - some of
them quite strong, but so far non have been greater than 4.7 - a
magnitude less than the main 0709 MDT shock.
For the most part, services (power, water, Internet)
have been minimally affected overall with the mobile phone service possibly having been impacted by the
expectedly-high call/text volume. Initial reports are that outages are restricted to very local cases where specific
sites may have had equipment disruption (e.g. loss of power/connectivity).
As far as can be determined, the Northern Utah WebSDR - which is about 70 miles (110 km)
north of Salt Lake City - saw no interruptions in power or service.
If there had been interruptions in service it would most likely
have been due to follow-on effects from widespread power failure and
loss of Internet connectivity in the nearby metro areas.
WebSDR Issues:
Wiring damage: It
was observed that some of the receivers' signal levels were low, so a
site visit was made where it was discovered that some of the wiring on
the 12 volt bus that powers the RF infrastructure (preamps, converters, etc.)
had been pulled apart - probably because of shaking of the equipment
and rack. There were a few brief outages as the DC cabling was
re-dressed and repaired.
USB port issues:
While on site it was observed that the system was throwing errors
related to the 40CW USB receiver, occasionally causing the WebSDR
service to restart which would kick everyone off. After a bit of
reconfiguring - which required some changes to the computer, WebSDR
reconfiguration, several reboots and 15-20 minutes of being offline -
that device was moved to a different USB port where it seems to be
behaving itself.
Dirty power on site:
A
few days ago a "new" UPS was put on site after the previous UPS had
been damaged by voltage extremes - and based on the complaints, the new
one was not happy at all with what it was seeing - with the AC mains
voltage varying from 90 to 142 volts - the UPS going on battery when it
was below about 105 volts or above 135 volts. In speaking with
the power company, we should not expect the situation to improve in the
short term which means that we will have to deal with it ourselves (power in rural areas is often like this) and we are considering more robust means of providing back-up power, including:
An AC mains voltage regulator
An
"online" UPS that can take a wide voltage input and output a consistent
125 volts. Essentially, this is would be just a wide
voltage-range 24 volt DC power supply connected to a battery and that
same battery connected to a 120 volt inverter.
If you have any suggestions - or equipment you might be willing to donate - please contact us as "sdrinfo@sdrutah.org".
KiwiSDR #2 intermittent:
KiwiSDR #2 has been having problems, occasionally rebooting. We think that this is a result of problem with the power lead feeding it and have effected repairs, but we are continuing to monitor.
Adjustments were made to the four "canned" settings of the DSP noise reduction (e.g. "Low", "Medium", "High", "Strong") to make them more usable overall:
Low
- This setting was adjusted to make it less agressive, but still
noticable. The idea is to take a bit of the edge off the noise
that might be present.
Medium
- This is roughly comparable to what the "low" setting had been:
It's effect is quite obvious, but likely not enough to be
annoying in most situation.
High
- The adaptation of the filter to voice is noticeably stronger, but it
will have more of an effect on background noise when no signals are
present (e.g. the "hollow" or "barrel" sound).
Strong - This
setting borders on the extreme. It will adapt fairly quickly to
voice energy buried in the noise and this has the effect of leaving a
bit of an "after image" (a ghostly echo) after the voice has stopped.
Remember:
This type of DSP noise reduction - like almost any other - "keys" on the voice energy among the noise, but this means that with very weak signals, there may not be enough voice energy to do this and it may actually reduce intelligibility in certain cases.
This type of filter works best with "white" noise (hiss): It will do nothing to help with off-frequency QRM (splatter) - because that is not noise!
Types of noise that are not random - such as powerline noise (e.g. buzz)- will not be reduced by the DSP noise reduction filter.
14 March, 2020: Extreme power events likely related to high winds in Northern Utah.
Likely
due to high winds, the electrical power at the Northern Utah WebSDR
receive site has been very erratic. It so-happened that one of
our number was at the receive site when a power bump occurred - but for
reasons to be determined, the UPS refused to take over. Of
course, the WebSDR and all of the on-site servers lost power when this
happened, which occurred at about 2008 MT. It took several
minutes for all WebSDR servers to come back online.
After
the power returned and stabilized, the UPS was tested repeatedly (main
power unplugged and plugged in) to replicate the issue, but the UPS
seemed to behave normally.
It
is suspected that some combination of brown-out or high line voltage
may have caused the UPS to freak out - but since the problem could not
be replicated, we may never know.
It was reported that there were scattered power outages in Corinne. In one location where the was power, the line voltage was seen to vary from zero volts to over 160 volts on a (nominal)
120 volt line for seconds at a time! At that particular location,
the UPS had disconnected itself from the power mains and was running
its load on battery-only during the wide voltage excursions.
13 March, 2020: Internet connectivity of the Northern Utah WebSDR affected by the Covid-19 health emergency.
What is happening: Our Internet Service Provider (ISP) has notified us that their connectivity (e.g. connectivity from the entities that provide back haul connectivity to large commercial entities and ISPs) has been restricted for the time-being. This is a follow-on effect caused by, among other things:
Increased use of bandwidth for tele-commuting as workplaces are encouraging employees to work remotely, where possible.
The temporary closure of educational institutions to in-person attendance and moving as many classes online as possible (e.g. remote learning).
Increased
use of home Internet connections for not only remote learning and
telecommuting, but also due to higher demand of online streaming (movie, TV) services because of more people staying home.
The ISP has (understandably) requested that their users minimize their bandwidth as much as practical.
As
you might expect, this sort of problem is affecting rural areas across
the U.S. as they typically have rather limited Internet connectivity,
anyway. The Northern Utah WebSDR is located in a rather
rural/remote area, so we are similarly affected.
What we have done at the Northern Utah WebSDR to reduce bandwidth utilization and maintain service:
Increased audio compression.
To reduce overall bandwidth, the amount of audio compression on
the users' streams has been increased: This will reduce audio
quality somewhat (slightly more noise and distortion) but the effects will likely not be noticed except for very good signals (e.g. "armchair" quality). This can be mitigated somewhat by setting the DSP Noise Reduction to the "Low" setting.
Reduced the inactivity timeout length.
The timeout timers have been reduced to 60 minutes to reduce
probability of "set and forget" monitoring: If nothing has been
done by the user (e.g. an adjustment of the WebSDR's settings) users will be disconnected after 60 minutes - but you will be able to reconnect.
Decreased waterfall speed. The rate of the waterfall scrolling will slow if a large number of users connects to the WebSDR.
Maximum number of users has been reduced. Reducing the maximum number of users will reduce the probability of bandwidth restrictions will affect all users of the Northern Utah WebSDR being affected if we hit the current, lower limit.
How long will this last?
Who's
to say? The increased use of Internet back hauls will be long
term and the Internet providers are ramping up capacity as much as
practical, but the increase in the worldwide demand of the very equipment needed to increase bandwidth (routers, interfaces, customer equipment, etc.)
at the very same time that staffing and supply line issues are
restricted will make this a formidable challenge - but hard work is
being done behind the scenes to reduce the impact
We thank you for your understanding in this matter - and please be safe!
11 March, 2020: Power failure
At
approximately 0915 MT the power at the site went offline and about 2
hours later, the UPS battery was exhausted. At approximately 1145
the power was restored, but apparently it did not do so cleanly:
The power supply in WebSDR #1 was damaged and although the other
servers powered up, they did not recover gracefully.
A
site visit - which occurred about 1600 MT along with more rebooting and
a swap of a power supply was able to bring everything back online by
about 1645.
We are still investigating the nature of the damage to the power supply and why it was that absolutely nothing
came back online by itself: We have had power failures before,
but we have never had so many things refuse to come back up unattended.
It appears that the LF receive system (reception below 400 kHz - including the 2200/1750M receiver)
was damaged at the moment of the original power failure - and did not
recover with the rest of the system: This is on the list of
things to be investigated during the next site visit.
Update:
A brief site visit was made and the power supply feeding this
antenna's coupler was repaired on 12 March, bringing it back online.
9 March, 2020: DSP Noise reduction and "Notch2" added to WebSDR #1 and #2:
The
DSP noise reduction and the second notch filter have been added to
WebSDR #1 and #2 after a few adjustments and bug fixes were made to
them from testing on WebSDR #3.
There
may still be some issues (bugs) with the new filters, so they should be
used with that in mind. If you notice what seems to be a bug -
particularly if you can reliably reporoduce it - please let us know via
email at "sdrinfo@sdrutah.org".
If,
while using the DSP noise reduction, the audio suddenly stops, try
turning the filter off and then back on in an attempt to reset it.
Suggested setting of DSP noise reduction for casual listening: The Low setting is recommended as it has a noticeable effect with the fewest artifacts.
7 March, 2020: DSP Noise reduction and additional notch filtering being tested on WebSDR #3 (blue):
Additional notch filter:
In addition to the original notch filter (had been labeled "Autonotch" - now labeled "Notch1"
another notch filter has been implemented. While the original
autonotch filter worked, it had some limitations due to the need to
minimize the processor load on the server: It was able to notch
only one tone at a time and it was easily "fooled" by modulation, the
result being that one could often hear the tone turning and and off as
the original notch filter "hunted". Because this notch filter is
at the server, activating it will cause the S-meter to drop when the
strong tones are removed.
The new filter (labeled "Notch2")
is a bit more agressive and should have less of a tendency to "hunt".
Because it is on the audio stream from the server, it will not
affect the S-meter, meaning that a strong "tuner-upper" will still
"de-sense" the receiver - unless "Notch1" is also active.
For
minor tones, Notch2, alone is fine, but it's just fine to use both.
Note that these filters will "hunt" a bit and can cause a bit of
odd distortion at times.
Of course, you should NOT use any notch filter if you are running CW or any digital mode!
DSP Noise reduction:
A
simple noise reduction system has been implemented and like "Notch2",
it is also audio-based, operating on the audio stream from the server.
There is a drop-down menu that allows several settings (including "Off") for the amount of noise reduction.
As
is typical of this type of noise reduction, it can have an "hollow"
sound - particularly with very noisy signals but that's just the way it
works.
These filters are designed for voice: They may sound terrible if you use them when listening to music - but try different settings.
The noise reduction works best on strong-to-"medium-weak" signals, but if a signal is extremely weak and noisy, it may not help: Go ahead a try the various settings.
In the presence of very weak (or no signals - just band noise) the audio may sound quiet - particularly if the noise reduciton is set high. Be careful when listening as you may be tempted to turn up the volume, only to get blasted when a strong signal appears.
This filter works best on "white" type noise, but it is less-effective on power-line buzz. The "Notch2" filter may work better in some cases.
These filters are currently being tested/debugged:
The settings of these filters is currently being tweaked and they may be improved over time.
Consider
these filters to be in "Alpha" testing: They may be buggy and may
not work as expected in some cases and there may be (yet unknown) compatibility issues. If you find what you think are issues with them drop a line to sdrinfo@sdrutah.org .
These
filters have been tested on Firefox and Chrome as those are the only
browsers currently available for testing. Reports of testing on
other browsers (e.g. Safari) and platforms other than PC are welcome.
It
has long been observed that the Firefox browser works better with
WebSDRs than Chrome: If you are a Chrome user and have issues it
is suggested that you also try Firefox.
If you are having issues with the Chrome browsers, see if your sound card settings (on Windows: Control Panel -> Sound -> Playback -> (select output device) -> Advanced)
and make sure that either "16 Bit 44100 Hz" or "16 Bit 48000 Hz" is
selected as Chrome doesn't seem to do well with higher rates (96000, 192000 Hz).
These filters require more processing horsepower when active than when they are off: This may pose issues (audio drop-outs, etc.) on older/slower computers. Some devices with limited processing power (phones, tablets) may have more problems.
These new features have not yet been rolled out to WebSDR #1 or #2 (or anywhere else) yet: When (and if!) depends on the results of testing.
4 March, 2020: FM demodulation modified.
It was suspected - and verified that the original "FM" demodulation of the WebSDR code did not implement audio de-emphasis. As you may be aware, in FM communications system the audio is not modulated as "straight" FM, but rather the audio is boosted (pre-emphasized) at 6dB/octave on transmit: In other words, given a constant audio level applied to the input of the modulation, the proper
implementation with pre-emphasis applied would be that this level might
give 1 kHz deviation with a 500 Hz tone, 2 kHz deviation with a 1000 Hz
tone and 4 kHz tone. On the receive end, the opposite should be
applied.
Without
de-emphasis, the audio of a typical "FM" signal sounded decidedly
"tinny" - which is to be expected - but it also sounded noisier than it
otherwise would. One of the reasons for transmit audio
pre-emphasis is because of the nature of FM: When a signal gets
weak, noise appears - but at the high frequencies, first. If you
boost the audio on the transmit end and "un-boost" it on the receive
end that noise is reduced by a significant amount.
If
de-emphasis is applied, low frequencies are effective boosted which can
cause subaudible tones to seem to be extremely loud, so high-pass
filtering was added as well: Three cascaded biquad filters - each
with a 12dB/octave roll-off below 225 Hz - pretty much eliminate any
the fundamental frequency of any subaudible tones that might be present
on the signal. Without this high-pass filtering such tones would
surely rattle the daylights out of the speaker!
These
filters were implemented inline in the main handler of the audio HTML5
javascript code so they will not add any additional processing delay.
This modification is most significant on WebSDR #3 which natively covers the 2 meter band.
25 February, 2020: Minor updates.
USB sound device port enumeration implemented:
On 11 January three FiFiSDR receivers were installed, replacing the previously-used (and ultumately unreliable)Asus
Xonar U5 and U7 USB cound cards. As noted, the FiFiSDRs are
complete units with both receiver and sound card hardware connected to
the computer by USB and this hardware has been successfully used by
several other WebSDRs - including the KFS (Half Moon Bay) WebSDR.
One
of the problems with identical USB devices - on both Linux and Windows
- is that it can be difficult to figure out which one is which:
If you have attempted to use multiple USB to Serial interfaces,
you will already be aware of this. With these sound cards (and the older Asus USB sound cards)
not only did the "name" of the sound card depend on into which USB port
it was plugged, but this naming could change if another USB device was
added.
An
addition - using Linux UDEV rules and a Perl script - was made so that
a sound device plugged into a particular USB port would always
have a consistent name. This Perl script was kindly provided by
Craig, W6DRZ, of the KFS WebSDR - and this script is a modified version
of one provided by "gmaruzz", described here.
The installation of this script required several reboots/restarts of WebSDR #1 (Yellow) to install and configure - some of these occurring in the early hours and another occurring mid-morning.
19 February, 2020: Service interruptions, upgrades.
There were several interruptions on this day while several pieces of gear (mostly networking)
were reconfigured and/or upgraded between 1000 and 2000 MT. This change should reduce the
likelihood of audio drop-outs caused by packet loss/jitter/delay on the
Internet connection that feeds the Northern Utah WebSDR's receive site.
This work caused a number of outages totalling several hours - something that was unavoidable.
16 February, 2020: Comments.
Power issues: At
approximately 0715 UTC the connectivity to the Northern Utah WebSDR
remote site was lost due to a power failure. Service was restored around 1935 UTC.
Note:
There will be an outage during the week of 16-22 February, 2020
for an equipment upgrade - the precise time/date yet to be determined.
160M Intermod reduced:
A
custom-made 160 meter band-pass filter was constructed and
placed in the 160 meter signal path, prior to any active devices. As noted in the 10 February
entry, the FiFiSDR was being overloaded by signals near the top end of
the AM broadcast band, these spurious signals (mostly) disappearing at night
when they went to low power.
This
filter was placed in series with the "160 meter" tap of the
multicoupler, which is, itself a filter - but apparently not good
enough to take care of the strong signals. The "new" filter
offers 4 dB of attenuation in the 160 meter band, but 20dB at 1700 kHz
and in excess of 40 dB below about 1600 kHz.
The new filter resolved the "new"
intermod issue related to the KiwiSDRs: There are some low-level intermodulation signals
- notably one at 1940 kHz, which is a mix of 1160 kHz (KSL AM, 50 kW)
and twice the frequency of another station on 1550 kHz (e.g. (2 x 1550)
- 1160 = 1940).
We have ruled out the RF chain as being the
source of this intermodulation by bypassing several pieces of equipment that preceded the new filter (main splitter/BCB filter, any amplification, all of the lightning protection, etc.). The source of the signals could
be the TCI-530 antenna, its tower or, most likely, the miles of rusty
barbed-wire fencing surrounding the site rectifying and re-radiating
IMD.
In
the process of diagnosing the intermod problem, we had to interrupt the
RF signal paths several times which affected all bands to some degree.
New amplifier in the KiwiSDR chain:
As
noted in the 10 February entry, an RF line amplifier in the KiwiSDR
signal path had become unstable and a different amplifier was temporarily placed inline - but that amplifier wasn't well-suited for very low frequency (VLF, LF) use as it suffered from low-level ingress of switching supply noise (from the computer gear) via its power lead.
A
new amplifier was constructed specifically for this task was placed into
service.
This amplifier is based on the
venerable 2N5109 transistor - a medium-power RF device designed for low
intermoduation distortion that is usable from audio through UHF.
This
amplifier has been specially designed to be useful from high audio
frequencies to at least 50 MHz allowing to cover the VLF-HF range
received by the KiwiSDR receivers.
Special
attention was made to the design to decouple/filter the DC input of the
amplifier so that low-level signals - particularly those below 100 kHz
- could not find their way in via that route.
The
new amplifier has greatly reduced the QRM from the DC line input that
had appeared at VLF and LF frequencies during the temporary use of the
amplifier installed on 8 February, and it has much greater signal
handling capability than the previously-used MMIC-based RF line
amplifier.
Power
Line noise:
There is an intermittent power
line related noise that occasionally shows up - and we are working to
resolve this issue - but the system's noise blanker seems to be able to
remove most of it. This noise is typically the worst on 80/75
meters, but it seems to disappear at night as the band opens up and the
ionospheric noise submerges it. Several poles have been
identified
Audio/system
drop-outs: There have occasionally
been periods where the audio will drop
and/or the waterfall will freeze briefly. While this is most
often a result of the user's computer (e.g. your
computer!)
getting busy, pre-empting the browswer's audio processing or congestion
on your Internet congestion - particularly if you have Internet via
Wireless, Satellite or phone
- see "Miscellaneous Quirks",
below for some causes and work-arounds.
A
"pop" in the audio and an accompanying change in signal level:
There appears to be an intermittent cross-connection on the
antenna between a feed and guy wire that can cause noise in the
received signals and levels to
fluctuate during very windy periods on site - particularly on the
lower (160,
80/75 and the 630M-AM-160M) bands. A
similar-sounding - but unrelated - issue (described below)
exists with the AM demodulator. Occasionally, this
seems to be accompanied by an increase in
intermodulation distortion from AM broadcast band signals - a problem
that can affect the 75 Meter and lower-frequency bands.
Occasional
outages: We are working to
"neaten up" the installation so we occasionally have to take something
off line to dress a cable, replace a connector, "permanentize" an
installation. Tasks like this will cause outages of several
minutes duration. Some work is also being done at one or more
of
the interim sites that provide the Internet connection that
occasionally result in outages.
Low-level
switching supply "birdies":
With lots of computers comes the possibility of lots of QRM
from
them! Fortunately, there are no "strong" birdies, but there
are
some that, if they happen to sweep across the frequency to which you
are listening, will cause some mild annoyance.
Miscellaneous
quirks (e.g. "It's
supposed to be that way!"):
If the WebSDR
is not on an active window on YOUR computer: If
the WebSDR is not the currently active
window on your desktop, the computer will give it lower priority and
this will make audio drop-outs/waterfall freezes more common.
The
same thing can happen if your computer is "busy" doing something else,
such as other programs, updates, incoming emails, etc. The
same is likely true
with other platforms (phone,
Mac, etc.)The
first thing that you should do if you experience drop-outs is to switch
to the window running the WebSDR. Don't forget
that your own
Internet connection/ISP can result in drop-out issues as well.
It
has been observed that on a typical Windows machine, if the processor
utilization is over 65-70%, you may experience occasional drop-outs due
to delays in the operating system providing processor time to the code
that produces the audio and draws the waterfall.
Waterfall not
updating when you switch to another window: When
the waterfall is not visible (that
is, you have switched to a window and moved the one with the WebSDR in
the background)
it will stop - which makes sense if you can't see it. What
this
means is that when you switch back to where the waterfall is again
visible there will often be a clear line of demarcation between what
the signals were when you switched away and when you switched back.
Occasional
burst of noise across the band correlated with strong signals:
This is sometimes the result of the noise blanker that is
active
on all bands being triggered by the peaks of the strongest signal(s) on
the band. This effect is most easily spotted on 40 meters
during
the daytime where there is one signal that is extremely strong - often
on the leading edge at the beginning of a transmission - and the band
is otherwise quiet. This effect should not be confused with
other
things that cause similar-looking artifacts, such as static crashes
from distant lighting storms or brief signals from ionospheric and
ocean wave profilers.
When
listening to AM, a sudden "pop" in the audio:
It's not actually supposed to do this, but there a "quirk" (bug?)
in the AM demodulator that can cause this to happen, occasionally.
It seems to happen mainly when the signal level changes very
quickly due to QSB (fading)
but is less likely to happen on very steady signals or those with only
very slow QSB. The only "fix" for this - if it becomes
annoying -
is to listen on USB or LSB and zero-beat the carrier.
The bands on
the "other" server aren't visible to me unless I go to that "other"
server: It is this way because there are several
independent
WebSDR servers.
If you notice some issues that are unrelated to those
listed abovefeel free to use
the contact info on the About
page to let
us know about it.
Additional information:
For general information about this WebSDR system -
including contact info - go to the about
page(link).
For technical information about this WebSDR system, go to
the technical
info
page(link).
For a list of questions and answers, visit the FAQ
page (link).
For more information about the WebSDR project in general -
including information about other WebSDR servers worldwide and
additional technical information - go to http://www.websdr.org
