Measurement Data Shapes and Missing Value Handling

Execution setup

Run import numpy as np and import matplotlib.pyplot as plt in a code cell before the practice code. Use np for array operations and plt for plots.

Learning objectives

Convert a measurement table into an analyzable array, checking missing values, units, and plots. Review only the Python syntax needed for this task.

One Sample and Multiple Samples

Starting conditions

The (hours,score) pairs for 3 students are (2,55),(4,70),(6,85). Time is in h.

Procedure

1. Make each sample a Dictionary and group them in a List.
2. Create separate Lists containing only hours and only scores, and calculate their means.

Results to retain

• A 3-row data table and per-field means.

records=[{'hours':2,'score':55},{'hours':4,'score':70},{'hours':6,'score':85}]
print([r['hours'] for r in records])

Record results and self-check

Samples:

Time [h]:

Score:

Data type:

Points to check

• Did you avoid confusing numbers with strings?

Samples with missing values

Starting conditions

The fourth sample has hours=3, score=None. Distinguish missing samples when calculating the mean.

Procedure

1. Add {"hours":3,"score":None} to records to create records2. Extract hours and score by field.
2. Calculate each field's mean, excluding only values that are None. In two rows, record field name, original value count, count used for the mean, and mean.
3. Compare the means before and after the addition, and explain whether a missing score is a reason to exclude its hours value as well.

Results to retain

• A table of per-field value counts and means before and after adding data.

records2=records+[{'hours':3,'score':None}]

Record results and self-check

Field name:

Total number of values:

Number of values used for the mean:

Means before and after the addition:

Points to check

• Did you distinguish the number of values used for each field's mean?

Making decisions with prepared data

Starting conditions

Organize the new device's records [(0,18),(1,19),(2,None)] using the same field names.

Procedure

1. Create a list of dictionaries from the new records, mapping the first value in each pair to the previous activity's input field and the second to its measurement field. Preserve None unchanged.
2. Calculate the count and mean of valid measurements, and plot inputs against measurements. Change one missing-value handling rule and calculate the same results again. For example, you can compare excluding missing values with replacing them with 0.
3. Record the two selected handling rules, valid-value counts, and means. Relate these results to why you need to understand what missing values mean in the original data.

Results to retain

• A scatter plot of the new data and a table of counts and means under each missing-value handling rule.

For the new device data, the first value is an input coordinate and the second is a measurement; physical units are not specified. Keep hours and score as keys for reusing the data structure, and label the plot axes 'Input coordinate' and 'Measurement'. Record the original observation count separately from the count used to calculate the mean after handling missing values.

# Change the conditions and check the results.

Record results and self-check

Two fields in the record:

Missing-value handling rule:

Original observation count, count used in calculation, and mean:

Scatter plot and interpretation:

Points to check

• Did you distinguish originally missing values from values after processing?