Data Analyser: CSV file details #sdr-consolev3


Nick Hall-Patch
 

This may be more a forum question, but isn't a bug report, more a "how is the sausage being made" question.

When I have created a CSV file in Data Analyser, it has taken its information from a +/- 0.5Hz slice of spectrum.

For each timestamp, is that data the total power level within the +/-0.5Hz bandwidth, or the maximum power level?

Thank you.

Nick


jdow
 

Um, er, ah, 1 Hz bandwidth? I am not sure SDRRadio will do that in any meaningful sense. You are pushing the limits of what a tool designed for picking olives when you try to use it to pick grapes. You may have to design your own instrument if you want that level of instrumentation accuracy. Depending on the front end the measurements you would get would be whimsical at best.

{o.o}

On 20210421 09:32:57, Nick Hall-Patch wrote:
This may be more a forum question, but isn't a bug report, more a "how is the sausage being made" question.

When I have created a CSV file in Data Analyser, it has taken its information from a +/- 0.5Hz slice of spectrum.

For each timestamp, is that data the total power level within the +/-0.5Hz bandwidth, or the maximum power level?

Thank you.

Nick


Nick Hall-Patch
 

SDRConsole's Data Analyser (in View / Options) does indeed make that claim, resolution bandwidth to better than 0.01Hz under the right settings, when playing back previously recorded SDR files.  More samples are used to generate finer scaled FFTs than are used in most SDR playback software.  See https://dk8ok.org/2021/01/22/magnificient-fdm-s3-the-millihertz-magnifier/ for example.

There is a possibility of generating CSV files of signal strength versus time for a passband of 1Hz (but only 1Hz) in Data Analyser....hence my question, because other software, such as Carrier Sleuth ( https://www.blackcatsystems.com/software/medium_wave_carrier_display_app.html ) can generate CSV files for narrower bandwidths yet, but is very clear that it is displaying the highest value signal strength within that bandwidth, unless you are asking for a single bin worth of bandwidth.  But peak value that can be a different, and lower, value from the total power within a passband.

73

Nick


At 23:57 2021-04-21, jdow wrote:
Um, er, ah, 1 Hz bandwidth? I am not sure SDRRadio will do that in any meaningful sense. You are pushing the limits of what a tool designed for picking olives when you try to use it to pick grapes. You may have to design your own instrument if you want that level of instrumentation accuracy. Depending on the front end the measurements you would get would be whimsical at best. {o.o}
On 20210421 09:32:57, Nick Hall-Patch wrote:
This may be more a forum question, but isn't a bug report, more a "how is the sausage being made" question. When I have created a CSV file in Data Analyser, it has taken its information from a +/- 0.5Hz slice of spectrum. For each timestamp, is that data the total power level within the +/-0.5Hz bandwidth, or the maximum power level? Thank you. Nick

Nick Hall-Patch
Victoria, BC
Canada


jdow
 

OK - I was just wondering.

Post processing I can see forcing the data to provide that level of bandwidth, at least in theory. I am not at all sure you can ask most SDRs or post processors to provide data in a precise 1Hz bandwidth. And even if it is one bin of the FFT (with a power of two sampling clock) that's not a precise 1 Hz noise bandwidth, I don't believe.

Are you looking to record the noise in a 1 Hz bandwidth or are you looking for signal plus noise with a signal that varies level (but not frequency?) The former seems possible with a correction factor. You may have to provide the factor by calibrating the system. The detection mode will make some difference on the readout. Is it a peak or average detector? Is it a true RMS detector? (I^2+Q^2 related.) Is it a rectifier and peak or average detector? All these have something to say for them. I prefer the true RMS approach. But then you have post filtering on the data which can upset calibration, particularly if signal plus noise is involved.  I don't know that Simon has released enough information to perform the analysis to get an analytical solution here. I remember some old HP Journal articles (yes, HP old) addressing this. Even the filter shapes will tweak your measurement of noise.

I always worked with known equipment and settings to create known Eb/N0 values. Fortunately I never had to work backwards and calculate Eb/N0 for a really weak signal in noise. That is a thoroughly fugly calculation with just a whole lot of things that can get in the way of accuracy. That is why I believe I'd calibrate the system myself.

And do be aware that you can specify to 1 Hz but, for example, your rtlsdr dongle will never be guaranteed to be accurate to that frequency within 7 Hz plus any clock error present. The synthesizers involved have their own step size which is NOT 1 Hz. That may be the source of another person's problems here; but, the description is not clear enough to know.

As a side issue, where do you get a signal that is stable enough to be seen in the noise of this kind of setup? Over the air signals vary considerably with small variances in pressure, temperature, and humidity even on ground wave paths. Through the ionosphere I suspect it might not be possible depending on the length of observation and the behavior of the Sun.

{O_O}

On 20210421 18:28:44, Nick Hall-Patch wrote:
SDRConsole's Data Analyser (in View / Options) does indeed make that claim, resolution bandwidth to better than 0.01Hz under the right settings, when playing back previously recorded SDR files.  More samples are used to generate finer scaled FFTs than are used in most SDR playback software.  See https://dk8ok.org/2021/01/22/magnificient-fdm-s3-the-millihertz-magnifier/ for example.

There is a possibility of generating CSV files of signal strength versus time for a passband of 1Hz (but only 1Hz) in Data Analyser....hence my question, because other software, such as Carrier Sleuth ( https://www.blackcatsystems.com/software/medium_wave_carrier_display_app.html ) can generate CSV files for narrower bandwidths yet, but is very clear that it is displaying the highest value signal strength within that bandwidth, unless you are asking for a single bin worth of bandwidth.  But peak value that can be a different, and lower, value from the total power within a passband.

73

Nick


At 23:57 2021-04-21, jdow wrote:
Um, er, ah, 1 Hz bandwidth? I am not sure SDRRadio will do that in any meaningful sense. You are pushing the limits of what a tool designed for picking olives when you try to use it to pick grapes. You may have to design your own instrument if you want that level of instrumentation accuracy. Depending on the front end the measurements you would get would be whimsical at best. {o.o}
On 20210421 09:32:57, Nick Hall-Patch wrote:
This may be more a forum question, but isn't a bug report, more a "how is the sausage being made" question. When I have created a CSV file in Data Analyser, it has taken its information from a +/- 0.5Hz slice of spectrum. For each timestamp, is that data the total power level within the +/-0.5Hz bandwidth, or the maximum power level? Thank you. Nick

Nick Hall-Patch
Victoria, BC
Canada



Nick Hall-Patch
 

For those who are interested, reading the couple of links provided below will give a better idea of what is going on, nothing to do with demodulation or rtlsdr dongles or suchlike things.   As for over the air signals, a number of AM broadcast signals do have stable carriers, even to within 0.1Hz.  Try Data Analyser and find out, though, yes, a decent SDR with a stable clock, likely disciplined by GPS, is needed to do the original recording.  Variations in the amplitude or frequency of the received signal can indicate propagation disturbances....that is the purpose of the exercise.

But, back to the original question...

73

Nick


At 01:59 2021-04-22, jdow wrote:
OK - I was just wondering. Post processing I can see forcing the data to provide that level of bandwidth, at least in theory. I am not at all sure you can ask most SDRs or post processors to provide data in a precise 1Hz bandwidth. And even if it is one bin of the FFT (with a power of two sampling clock) that's not a precise 1 Hz noise bandwidth, I don't believe. Are you looking to record the noise in a 1 Hz bandwidth or are you looking for signal plus noise with a signal that varies level (but not frequency?) The former seems possible with a correction factor. You may have to provide the factor by calibrating the system. The detection mode will make some difference on the readout. Is it a peak or average detector? Is it a true RMS detector? (I^2+Q^2 related.) Is it a rectifier and peak or average detector? All these have something to say for them. I prefer the true RMS approach. But then you have post filtering on the data which can upset calibration, particularly if signal plus noise is involved.  I don't know that Simon has released enough information to perform the analysis to get an analytical solution here. I remember some old HP Journal articles (yes, HP old) addressing this. Even the filter shapes will tweak your measurement of noise. I always worked with known equipment and settings to create known Eb/N0 values. Fortunately I never had to work backwards and calculate Eb/N0 for a really weak signal in noise. That is a thoroughly fugly calculation with just a whole lot of things that can get in the way of accuracy. That is why I believe I'd calibrate the system myself. And do be aware that you can specify to 1 Hz but, for example, your rtlsdr dongle will never be guaranteed to be accurate to that frequency within 7 Hz plus any clock error present. The synthesizers involved have their own step size which is NOT 1 Hz. That may be the source of another person's problems here; but, the description is not clear enough to know. As a side issue, where do you get a signal that is stable enough to be seen in the noise of this kind of setup? Over the air signals vary considerably with small variances in pressure, temperature, and humidity even on ground wave paths. Through the ionosphere I suspect it might not be possible depending on the length of observation and the behavior of the Sun. {O_O}



On 20210421 18:28:44, Nick Hall-Patch wrote:
SDRConsole's Data Analyser (in View / Options) does indeed make that claim, resolution bandwidth to better than 0.01Hz under the right settings, when playing back previously recorded SDR files.  More samples are used to generate finer scaled FFTs than are used in most SDR playback software.  See https://dk8ok.org/2021/01/22/magnificient-fdm-s3-the-millihertz-magnifier/ for example. There is a possibility of generating CSV files of signal strength versus time for a passband of 1Hz (but only 1Hz) in Data Analyser....hence my question, because other software, such as Carrier Sleuth ( https://www.blackcatsystems.com/software/medium_wave_carrier_display_app.html ) can generate CSV files for narrower bandwidths yet, but is very clear that it is displaying the highest value signal strength within that bandwidth, unless you are asking for a single bin worth of bandwidth.  But peak value that can be a different, and lower, value from the total power within a passband. 73 Nick At 23:57 2021-04-21, jdow wrote:
Um, er, ah, 1 Hz bandwidth? I am not sure SDRRadio will do that in any meaningful sense. You are pushing the limits of what a tool designed for picking olives when you try to use it to pick grapes. You may have to design your own instrument if you want that level of instrumentation accuracy. Depending on the front end the measurements you would get would be whimsical at best. {o.o} On 20210421 09:32:57, Nick Hall-Patch wrote:
This may be more a forum question, but isn't a bug report, more a "how is the sausage being made" question. When I have created a CSV file in Data Analyser, it has taken its information from a +/- 0.5Hz slice of spectrum. For each timestamp, is that data the total power level within the +/-0.5Hz bandwidth, or the maximum power level? Thank you. Nick

Nick Hall-Patch Victoria, BC Canada

Nick Hall-Patch
Victoria, BC
Canada


jdow
 

Nick supplied me with some numbers. He is interested in a small frequency shift, under 100Hz, when retuning the front end, the AirSpy R2. The AirSpy R2 uses an R820T2 front end chip. It's frequency synthesizer step is fairly large. It is large enough that rtlsdr.dll recognizes this chip and uses the IF frequency synthesizer to make corrections so that the error is less than 7Hz. I do not believe at least some versions of AirSpy R2 software do not use it's internal synthesizer capability to tune out the R820T2's synthesizer's step size.

I also suggested he'd probably be better off leaving SDRC tuned to 470.0 MHz and let the receiver offset tune the 310 kHz offset to 470.310 MHz. It might avoid R820T2 synthesizer self conversion noise. And the receive windows can give him a better resolution view. (If he'd like even more I'd suggest he try SDRSharp. When zooming it leaves tuning the same but moves that receive frequency to the middle of the screen inside the display software.)

{^_^}

On 20210421 22:04:08, Nick Hall-Patch wrote:
For those who are interested, reading the couple of links provided below will give a better idea of what is going on, nothing to do with demodulation or rtlsdr dongles or suchlike things.   As for over the air signals, a number of AM broadcast signals do have stable carriers, even to within 0.1Hz.  Try Data Analyser and find out, though, yes, a decent SDR with a stable clock, likely disciplined by GPS, is needed to do the original recording.  Variations in the amplitude or frequency of the received signal can indicate propagation disturbances....that is the purpose of the exercise.

But, back to the original question...

73

Nick


At 01:59 2021-04-22, jdow wrote:
OK - I was just wondering. Post processing I can see forcing the data to provide that level of bandwidth, at least in theory. I am not at all sure you can ask most SDRs or post processors to provide data in a precise 1Hz bandwidth. And even if it is one bin of the FFT (with a power of two sampling clock) that's not a precise 1 Hz noise bandwidth, I don't believe. Are you looking to record the noise in a 1 Hz bandwidth or are you looking for signal plus noise with a signal that varies level (but not frequency?) The former seems possible with a correction factor. You may have to provide the factor by calibrating the system. The detection mode will make some difference on the readout. Is it a peak or average detector? Is it a true RMS detector? (I^2+Q^2 related.) Is it a rectifier and peak or average detector? All these have something to say for them. I prefer the true RMS approach. But then you have post filtering on the data which can upset calibration, particularly if signal plus noise is involved.  I don't know that Simon has released enough information to perform the analysis to get an analytical solution here. I remember some old HP Journal articles (yes, HP old) addressing this. Even the filter shapes will tweak your measurement of noise. I always worked with known equipment and settings to create known Eb/N0 values. Fortunately I never had to work backwards and calculate Eb/N0 for a really weak signal in noise. That is a thoroughly fugly calculation with just a whole lot of things that can get in the way of accuracy. That is why I believe I'd calibrate the system myself. And do be aware that you can specify to 1 Hz but, for example, your rtlsdr dongle will never be guaranteed to be accurate to that frequency within 7 Hz plus any clock error present. The synthesizers involved have their own step size which is NOT 1 Hz. That may be the source of another person's problems here; but, the description is not clear enough to know. As a side issue, where do you get a signal that is stable enough to be seen in the noise of this kind of setup? Over the air signals vary considerably with small variances in pressure, temperature, and humidity even on ground wave paths. Through the ionosphere I suspect it might not be possible depending on the length of observation and the behavior of the Sun. {O_O}



On 20210421 18:28:44, Nick Hall-Patch wrote:
SDRConsole's Data Analyser (in View / Options) does indeed make that claim, resolution bandwidth to better than 0.01Hz under the right settings, when playing back previously recorded SDR files.  More samples are used to generate finer scaled FFTs than are used in most SDR playback software.  See https://dk8ok.org/2021/01/22/magnificient-fdm-s3-the-millihertz-magnifier/ for example. There is a possibility of generating CSV files of signal strength versus time for a passband of 1Hz (but only 1Hz) in Data Analyser....hence my question, because other software, such as Carrier Sleuth ( https://www.blackcatsystems.com/software/medium_wave_carrier_display_app.html ) can generate CSV files for narrower bandwidths yet, but is very clear that it is displaying the highest value signal strength within that bandwidth, unless you are asking for a single bin worth of bandwidth.  But peak value that can be a different, and lower, value from the total power within a passband. 73 Nick At 23:57 2021-04-21, jdow wrote:
Um, er, ah, 1 Hz bandwidth? I am not sure SDRRadio will do that in any meaningful sense. You are pushing the limits of what a tool designed for picking olives when you try to use it to pick grapes. You may have to design your own instrument if you want that level of instrumentation accuracy. Depending on the front end the measurements you would get would be whimsical at best. {o.o} On 20210421 09:32:57, Nick Hall-Patch wrote:
This may be more a forum question, but isn't a bug report, more a "how is the sausage being made" question. When I have created a CSV file in Data Analyser, it has taken its information from a +/- 0.5Hz slice of spectrum. For each timestamp, is that data the total power level within the +/-0.5Hz bandwidth, or the maximum power level? Thank you. Nick

Nick Hall-Patch Victoria, BC Canada

Nick Hall-Patch
Victoria, BC
Canada