Sequence Data and Time

Goal

By the end of this lesson, you can explain why order is part of sequence meaning, compare order-aware and order-blind features, and identify a future-information leakage bug.

The same values can tell opposite stories

Compare two temperature sequences:

[1, 2, 5]

and

[5, 2, 1]

They contain exactly the same numbers. Their arithmetic mean is the same:

(1 + 2 + 5) / 3 = 8/3

But the first sequence rises and the second falls.

If your task depends on direction, a mean throws away the information you need.

A sequence is therefore not just a collection of values. Position and order can be part of the input's meaning.

Different summaries preserve different information

For a sequence, you might compute:

• mean — ignores order;
• first value — keeps one endpoint;
• last value — keeps the other endpoint;
• last - first — keeps a crude direction;
• recent-window features — preserve some local order.

No summary is automatically best. The right representation depends on the prediction question.

Compare an original sequence with its reversal

The Lab computes order-aware and order-blind features.

1. Click Run. The starter sequence is [1., 2., 5., 9.], and the Lab also builds its reversal, [9., 5., 2., 1.].
2. Compare the forward: and reversed: lines.
3. Confirm that mean stays exactly 4.25. The average does not care about order.
4. Compare the order-aware features: direction (last minus first) flips from 8.0 to -8.0, and last_change changes from 4.0 to -1.0.
5. Now keep the same four numbers but shuffle them. Change sequence = np.array([1., 2., 5., 9.]) to sequence = np.array([1., 9., 2., 5.]). Before running, predict which features will still match the original run.
6. Click Run. mean is still 4.25, but forward now shows direction: 4.0 and last_change: 3.0. Same values, different order, different story.
7. Press Reset afterward.

Loading lab…

The experiment makes one thing visible: a representation can contain all the same numerical values and still lose the temporal relationship between them.

Sorting can be a silent sequence bug

Imagine a preprocessing step that sorts every sensor sequence from smallest to largest.

The code is deterministic and produces a tidy array. It also destroys the original time order.

For some tasks that may be harmless; for trend, language, audio, or event-sequence tasks it can destroy the signal.

Always inspect timestamps or positions before assuming preprocessing preserved sequence meaning.

Time creates a special leakage risk

Suppose you want to predict whether a machine will fail at 10:00 AM.

A feature computed from sensor readings at 10:30 AM contains information from the future relative to the prediction moment.

Even if the feature improves test accuracy, it is invalid for a prediction that must happen at 10:00.

Sequence evaluation therefore needs both correct row separation and correct time direction.

A sequence carries order-dependent information

Consider:

temperature: 10, 11, 12, 20

and the same values reordered:

20, 12, 11, 10

A bag of values contains the same numbers, but the time story is different.

Sequence models therefore need to preserve more than membership. They need to represent which observation came before which.

The prediction target determines which direction is legal

For next-step forecasting, predicting time t+1 should use information available at or before time t, not measurements from the future.

This creates the same kind of leakage boundary you saw in train/test splitting, now inside time.

A random split across individual time points can accidentally put later information into training while earlier points are used for evaluation.

Whenever data has temporal order, ask: what information would have actually existed at prediction time?

Quick Check

1. What can be lost if you sort a time sequence?

The sequence length, because sorting groups equal values togetherOriginal order informationDuplicate values, because sorting preserves the relative order of distinct values

2. Which summary ignores order?

First valueLast-minus-firstThe mean of all values

3. What is a common sequence leakage risk?

Using future information to predict the pastKeeping timestamps as metadata even when they are available at prediction timeUsing the same feature values at several adjacent timestamps

0 of 3 questions answered.

Key Takeaways

• Sequence order can carry information that the values alone do not.
• Order-blind summaries such as a mean can be useful but insufficient.
• Preprocessing can be numerically correct while destroying time meaning.
• Sequence features must respect the real prediction moment.

Next Lesson

Next, you will let a recurrent model carry a small hidden state from one sequence step to the next.

References

• PyTorch, nn.RNN.