Jump to content
  • 0
Alexey

QVRT Context packet Metadata Timestamp

Question

Let me describe the issue that I have. There is a file which contains data from both tuners and TunerA sample rate is 42 MS/s and TunerB is 13.3 MS/s. The app dumps the processed packets number and encountered metadata (WbtVRTFileReadIterator::getCurrentMetadataAsJSON) to debug output. I start processing TunerA data. The first metadata has "GPS Fix UTC Seconds" value of 1446218004. By the time app processed 815M samples (which should constitute approx. 19 seconds of data) the metadata timestamp is 1446218019 - only 15 seconds later than the starting value.

The difference is even bigger in case of TunerB. After processing 250M samples (~18 sec.) the metadata timestamp is 1446218009 - moved forward by only 5 sec.

What is happening here and is it the expected behavior?

Share this post


Link to post
Share on other sites

12 answers to this question

Recommended Posts

  • 0

I would say that this is indeed not the expected behavior. Does this happen consistently, or just with some files?

The things that come to mind that could cause this are:

1. Loss of GPS sync

2. Issues traversing file with file read iterator.

Two things that would help us get to the bottom of this would be either:

a) a source code snippet

b) versions of software where file is captured and version of API used

 

 

Share this post


Link to post
Share on other sites
  • 0
  1. The SingleTuner and Combined configs are fine. It happens only with DualTuner files. And then it correlates with tuners sample rates - if the sample rates are equal (2.2 + 2.2 MS/s) then each stream is ahead of metadata by the same seconds number; if the rates are different (0.27 + 13.3) then the stream with higher SR is closer to metadata timestamp. It look like the metadata packets are placed every [TunerA SR + TunerB SR] sample for every stream instead of [TunerA SR] for A stream and [TunerB SR] for B stream.
  2. GPS sync is perfect: every metadata shows "GPS Fix Status": "3D Fix".
  3. Hardware info (from file metadata): "Unit Hardware Version": "200", "Firmware Version": "2.4.6.34", "WBT File Format Version": "2.1". App is built with API v2.4.6.13.
  4. Code looks like this:
    void process(WbtCommon::Data::FileInfo file)
    {
        WbtApi::WbtVRTFileReadIterator iter(file._fileName, WbtCommon::Data::Radio_A);
        const int samplesPerSecond = file._radioSettings._radioA._span * 1e6L;
        const int valuesPerChunk = 33 * 995 * 2;
        qint64 totalSamples = 0;
        float *buffer = new float[valuesPerChunk];
        while(!iter.isEndOfStream())
        {
            switch(iter.getCurrentPacketType())
            {
                case VRT_IF_DATA:
                    totalSamples += iter.getCurrentIQDataFloat(buffer, valuesPerChunk);
                    foobar(buffer, valuesPerChunk);
                    break;
            }
            const int seconds = totalSamples / samplesPerSecond;
            if(iter.metadataChanged())
            {
                qDebug() << totalSamples << seconds << QString::fromStdString(iter.getCurrentMetadataAsJSON());
            }
            ++iter;
        }
        delete []buffer;
    }

Share this post


Link to post
Share on other sites
  • 0

Thank you for the additional information. We're looking into the issue and will reply with additional information soon. 

Share this post


Link to post
Share on other sites
  • 0

The following is sample code for getting samples for a combined tuner QVRT file. 

 

void process(WbtCommon::Data::FileInfo file)
{

    WbtVRTFileReadIterator iteratorA(file._fileName, WbtCommon::Data::Radio_A);
    WbtVRTFileReadIterator iteratorB(file._fileName, WbtCommon::Data::Radio_B);

    const int samplesPerSecond = file._radioSettings._combined._span * 1e6L;
    const int valuesPerChunk = 33*995*2;
    qint64 totalASamples = 0;
    qint64 totalBSamples = 0;

    float* buffer = new float[valuesPerChunk];

    while(iteratorA.isValid() && !iteratorA.isEndOfStream())
    {
        if(iteratorA.getCurrentPacketType() == VRT_IF_DATA)
        {
            totalASamples += iteratorA.getCurrentIQDataFloat(buffer, valuesPerChunk);
            
            //Do work with buffer data
        }
        iteratorA++;
    }

    while(iteratorB.isValid() && !iteratorB.isEndOfStream())
    {
        if(iteratorB.getCurrentPacketType() == VRT_IF_DATA)
        {
            totalBSamples += iteratorB.getCurrentIQDataFloat(buffer, valuesPerChunk);
            
            //Do work with buffer data
        }
        iteratorB++;
    }

    int combinedSeconds = (totalASamples + totalBSamples) / samplesPerSecond;

    delete[] buffer;

}

Share this post


Link to post
Share on other sites
  • 0

Won't it violate the temporal integrity if I read all A samples and then all B? Also, if I need to process only part of the file (that is, need to stop after certain amount of samples) - it would read some A packets and none of B packets. Is the following code more correct:

while(...) {

    ...

    totalSamples += iteratorA.getCurrentIQDataFloat(buffer, valuesPerChunk);

    // process buffer data

    totalSamples += iteratorB.getCurrentIQDataFloat(buffer, valuesPerChunk);

    // process

    iteratorA++;

    iteratorB++;

    int combinedSeconds = totalSamples / samplesPerSecond;

    if(combinedSeconds >= desiredDuration) break;

}

 

Also, is it safe to read metadata from only one of the iterators? Will it give correct steady data?

Share this post


Link to post
Share on other sites
  • 1

If you would like to iterate over a specific amount of time, I would do the following:

while(...) {

    ...

while(iteratorA.getCurrentPacketType() != VRT_IF_DATA && !iteratorA.isEndOfStream())

{

iteratorA++;

}

while(iteratorB.getCurrentPacketType() != VRT_IF_DATA && !iteratorB.isEndOfStream())

{

iteratorB++;

}

    totalSamples += iteratorA.getCurrentIQDataFloat(buffer, sampleRateA * desiredDuration * 2, WbtVRTFileReadIterator::Wbt_Samples_Continued_From_Last);

    // process buffer data

    totalSamples += iteratorB.getCurrentIQDataFloat(buffer, sampleRateB * desiredDuration * 2, WbtVRTFileReadIterator::Wbt_Samples_Continued_From_Last);

    // process

}

 

This will allow you to keep the temporal integrity. Wbt_Samples_Continued_From_Last will return all the samples you need for each time span. The sample rate needs to be checked for each tuner or else you can get out of time.

 

As far as the meta data, it depends on what you need from it. Does the information in one tuner's meta data contain the information you need?

Edited by Alex D
Changes

Share this post


Link to post
Share on other sites
  • 0

Sorry, I still don't get it. If I understand correctly, tuners record simultaneously during Combined config and the packets go like this: [A B A B ...], and I need to read them in the same order, not [A A ...] [B B ...]. Am I right?

Share this post


Link to post
Share on other sites
  • 1

You are correct in the fact that the packets can go [ A B A B] or any combination. For example [A A B A] is valid depending on the file. This is due to the fact that tuners A and B may not be set at the same sample rate causing more samples for one or the other in the same time frame.

 

Each iterator is independent of the other and only handles packets assigned to that particular stream. If you read one packet for A and then one packet for B, if the sample rates are different, you will become out of time. In order to make sure the file iterators stay in time, you must read samples in relation to the specific radio's sample rate.

 

For example, if radio A has a sample rate of 10MS/s and radio B has a sample rate of 20MS/s for a combined sample rate of 30MS/s, you would need to read 10,000,000 samples for the iteratorA and 20,000,000 samples for the iteratorB to read in 1 second of data. These values can be reduced to meet your needs but, the ratio of samples must stay consistent. 

 

I hope this helps clear up how the QVRT files work.

Share this post


Link to post
Share on other sites
  • 0

I examined several files with Combined tuners, looked into their "_radioSettings" variable. In every file the sample rate of each tuner was exactly 1/2 of combined sample rate, for example [A:42, B:42, AB:84], [A:41.6667, B:41.6667, AB:83.3333], [A:25, B:25, AB:50], [A:26.2417, B:26.2417, AB:52.4833].

But you're saying it's not strictly defined and I can't rely on this?

Share this post


Link to post
Share on other sites
  • 0

With combined tuners, this is usually the case, however, you can have situations where you have two separate streams of two separate sample rates interleaved in one file. In these situations, you would not be able to rely on this assumption.

For the combined case, your example would indeed work. But not for all configurations that use Radio_AB as a radio selection. 

Share this post


Link to post
Share on other sites
  • 0

Currently I'm only interested in Combined Radio_AB case, as the app I'm working on works only with Single or Combined. I would not want to make things too complicated, so if I can safely assume that both streams run with the same speed it would be great.

I'll keep in mind that DualTuner Radio_AB case is more complicated.

Thank you all for the explanations!

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×