Dynamically modified on flyΒΆ

If input data structure is complex, and only several fields take part in transformation, presaved data can be read, and fields modified on fly.

@Test
public void testMass_WhenWeightSpecified_ThenWeightsSum() {
    int expected = 85;
    Dataset<Row> input = getBulkData().withColumn("weight", lit(85));
    assertEquals(expected, repository.mass(input));
}

@Test(expected = Exception.class)
public void testMass_WhenWeightIsNull_ThenNPE() {
    Dataset<Row> input = getBulkData().withColumn("weight", lit(null).cast(DataTypes.IntegerType));
    repository.mass(input);
}

private Dataset<Row> getBulkData() {
    return spark().read().option("header", "true").csv(getTestDataFolder());
}

Example: DynamicallyModifiedOnFlyTest.java