Add validations on MCAS performance levels and other fields

[kevinrobinson]

There are many values in there beyond normal academic scores taht I didn't expect, including: PRG, TRN, MED, ABS, INP. I didn't investigate how this impacts charts or tables of these scores in the product to understand how meaningful this is.

[kevinrobinson]

Relatedly, here's some older notes on MCAS validations from StudentAssessment. These were from July 2018 but are stale now. Just capturing them here since I'm removing them as code comments in #2160:

  # Notes on data quality for MCAS:
    # Looking at Somerville raw data July 2018:
    #   * ~40% of rows have no growth percentile.
    #   * ~5% of rows have no scale score.
    #   * Almost all rows have performance level (only 1 exception).
    # Looking at New Bedford raw data July 2018:
    #   * Zero rows have growth percentile.
    #   * 33% of rows have no scale score.
    #   * 16% of rows have no performance level.

This is

[kevinrobinson]

Relatedly, here's some older notes on MCAS validations from StudentAssessment. These were from July 2018 but are stale now. Just capturing them here since I'm removing them as code comments in #2160:

  # Notes on data quality for MCAS:
    # Looking at Somerville raw data July 2018:
    #   * ~40% of rows have no growth percentile.
    #   * ~5% of rows have no scale score.
    #   * Almost all rows have performance level (only 1 exception).
    # Looking at New Bedford raw data July 2018:
    #   * Zero rows have growth percentile.
    #   * 33% of rows have no scale score.
    #   * 16% of rows have no performance level.

[kevinrobinson]

This could use doing, but needs production access. Helpful tidbit:

  Assessment.all.each do |assessment|
      puts "\n\nAssessment: #{assessment.family}, #{assessment.subject}"
      records = StudentAssessment.where(assessment_id: assessment)
      puts "records=#{records.size}"
      fields = [:scale_score, :growth_percentile, :performance_level, :percentile_rank]
      fields.each do |field|
        all_values = records.map {|r| r.send(field) }
        values = all_values.compact.uniq
        puts "  #{field}: unique=#{values.size}, nils=#{all_values.select(&:nil?).size}  #{values.min} - #{values.max}"
      end
    end
    nil