Allowing DateFormat in the parser

[venkatd]

In our application, depending on the locale of the user, we have a DateFormat that varies by the user (intl library). For example:

final kMessageDateTime = DateFormat('E MMM d, h:mm a');

I was considering to build actual grammars from these format tokens but that feels like overkill as there could be a bunch depending on the country of the user.

How might I implement something like the following?

final localDateTime = formattedDateTime('E MMM d, h:mm a');
final dates = localDateTime.matchesSkipping(someMessageWithTimeReferences);

From looking through the source, it seems like I would need to extend Parser and expose a helper method. If that's the case, any ideas on where I can begin? Any good examples I can look at?

Somewhat related... I see some parsers implement a fastParseOn and a parseOn. Why is parseOn necessary if fastParseOn is implemented?

[venkatd]

I'm torn between generating a grammar based on a DateFormat object vs. creating a custom parser.

For example, the following seems like it could work. (Hardcoded to just one case)

// hardcoded to just "E MMM d, h:mm a" to test things out
Parser<DocumentToken> dateTime(DateFormat format) {
  final shortMonth = anyWordIgnoringCase(format.dateSymbols.SHORTMONTHS);
  final dateMday = (num2Digit(min: 10, max: 31) | pattern('1-9')).flatten();
  final hour12 = (num2Digit(min: 1, max: 12) | pattern('0-9')).flatten();
  final amPm = anyWordIgnoringCase(format.dateSymbols.AMPMS);
  final timeMinute = num2Digit(max: 59);
  return (shortMonth &
          char(' ') &
          dateMday &
          string(', ') &
          hour12 &
          char(':') &
          timeMinute &
          char(' ') &
          amPm)
      .flatten()
      .map((value) => StringObject(string: value));
}

It looks like I could read from DateFormat.dateSymbols and map each format element (E MMM, d, h:mm, a, etc) to a corresponding parser. But it seems like a lot of code and unsure if it's cleaner to extend Parser and call DateFormat.parse.

[renggli]

You could create a parser that builds another parser relatively easy:

final day = 'dd'.toParser().map((token) => digit()
    .repeat(2)
    .flatten()
    .map((value) => MapEntry(#day, int.parse(value))));
final month = 'mm'.toParser().map((token) => digit()
    .repeat(2)
    .flatten()
    .map((value) => MapEntry(#month, int.parse(value))));
final year = 'yyyy'.toParser().map((token) => digit()
    .repeat(4)
    .flatten()
    .map((value) => MapEntry(#year, int.parse(value))));

final spacing = whitespace().map((token) =>
    whitespace().star().map((value) => const MapEntry(#unused, 0)));
final verbatim = any().map(
    (token) => token.toParser().map((value) => const MapEntry(#unused, 0)));

final entries = [day, month, year, spacing, verbatim].toChoiceParser();
final format = entries
    .star()
    .end()
    .map((parsers) => parsers.toSequenceParser().map((entries) {
          final arguments = Map.fromEntries(entries);
          return DateTime(
            arguments[#year] ?? DateTime.now().year,
            arguments[#month] ?? DateTime.january,
            arguments[#day] ?? 1,
          );
        }));

Then you can create date formats like so:

final iso = format.parse('yyyy-mm-dd').value;
final date = iso.parse('1980-06-11').value;
print(date); // 1980-06-11 00:00:00.000

I do not really see why you want to subclass your own Parser? fastParseOn is an optimized version of parseOn, that does not produce a result, but can provide a significant speed boost in some cases. Normally you should only need to implement parseOn.

[venkatd]

Thanks, I tweaked the code a bit and will post what I come up with later. I think because I have implicit-casts: false enabled in my analysis_options.yml, I had to add some explicit casting cast and castList. Otherwise the functions are returning List<dynamic>

I think if the extensions had generics added, these casting steps wouldn't have to be added. Is this something that's worth improving? I could create a separate issue to track if it seems worth while.

enum _DateTimeField { month, day, year, literal }

Parser<DateTime> dateTimeFromFormat(String format) {
  final day = 'dd'.toParser().map((token) => digit()
      .repeat(2)
      .flatten()
      .map((value) => MapEntry(_DateTimeField.day, int.parse(value))));

  final month = 'mm'.toParser().map((token) => digit()
      .repeat(2)
      .flatten()
      .map((value) => MapEntry(_DateTimeField.month, int.parse(value))));

  final year = 'yyyy'.toParser().map((token) => digit()
      .repeat(4)
      .flatten()
      .map((value) => MapEntry(_DateTimeField.year, int.parse(value))));

  final spacing = whitespace().map((token) => whitespace()
      .star()
      .map((value) => const MapEntry(_DateTimeField.literal, 0)));

  final verbatim = any().map((token) => token
      .toParser()
      .map((value) => const MapEntry(_DateTimeField.literal, 0)));

  final entries = [day, month, year, spacing, verbatim]
      .toChoiceParser()
      .cast<Parser<MapEntry<_DateTimeField, int>>>();

  final dtParser = entries.star().end().map((parsers) {
    return parsers
        .toSequenceParser()
        .castList<MapEntry<_DateTimeField, int>>()
        .map((entries) {
      final arguments = Map.fromEntries(entries);
      return DateTime(
        arguments[_DateTimeField.year] ?? DateTime.now().year,
        arguments[_DateTimeField.month] ?? DateTime.january,
        arguments[_DateTimeField.day] ?? 1,
      );
    });
  });

  return dtParser.parse(format).value;
}

[renggli]

Yeah, I already noticed myself that the choice parser was missing a generic type. Fixed this in b5e85d6. I was on the nullsafety branch, so that might explain some of the other differences. If you have any other suggestions of improving the types feel free to add bugs/pull-requests.

[venkatd]

Great! Will do. If I see any possibilities for improving the types will submit some issues and attempt a PR.

Here is what I have so far, maybe as reference to others looking into this library. I'm passing in a DateFormat class here so I can lean on the intl library to get things like month names, days of week, etc.

import 'package:intl/intl.dart';
import 'package:meta/meta.dart';
import 'package:petitparser/petitparser.dart';
import 'util.dart';

Parser<DateTime> dateTimeFromFormat(DateFormat format) {
  final num4Digits = digit().repeat(4).flatten().map(int.parse);

  final yyyy = string('yyyy').map((token) => num4Digits.map(_year));
  final y = string('y').map((_) => num4Digits.map(_year));
  final MMM = string('MMM').map((_) {
    return anyWordIgnoringCase(format.dateSymbols.SHORTMONTHS).map(
      (v) => _month(format.dateSymbols.SHORTMONTHS.indexOf(v) + 1),
    );
  });
  final MM = string('MM')
      .map((_) => number2Digit(min: 1, max: 12).map((v) => _month(v)));
  final mm = string('mm').map((_) => number2Digit(max: 59).map(_minute));
  final dd = string('dd').map((_) => number2Digit(min: 1, max: 31).map(_day));
  final d = string('d').map((_) => number(min: 1, max: 31).map(_day));
  final h = string('h').map((_) => number(min: 1, max: 12).map(_hour));
  final a = string('a').map((_) {
    return anyWordIgnoringCase(format.dateSymbols.AMPMS).map(
      (v) => MapEntry(_DateTimeField.ampm, format.dateSymbols.AMPMS.indexOf(v)),
    );
  });

  final spacing = whitespace().map((token) => whitespace()
      .star()
      .map((value) => const MapEntry(_DateTimeField.literal, 0)));

  final verbatim = any().map((token) => token
      .toParser()
      .map((value) => const MapEntry(_DateTimeField.literal, 0)));

  final entries = [yyyy, y, MMM, MM, mm, dd, d, h, a, spacing, verbatim]
      .toChoiceParser()
      .cast<Parser<MapEntry<_DateTimeField, int>>>();

  final dtParser = entries.star().end().map((parsers) {
    final now = DateTime.now();
    return parsers
        .toSequenceParser()
        .castList<MapEntry<_DateTimeField, int>>()
        .map((entries) => _fromFields(Map.fromEntries(entries), now: now));
  });

  return dtParser.parse(format.pattern).value;
}

enum _DateTimeField { year, month, day, hour, minute, ampm, literal }

MapEntry<_DateTimeField, int> _year(int y) => MapEntry(_DateTimeField.year, y);
MapEntry<_DateTimeField, int> _month(int m) =>
    MapEntry(_DateTimeField.month, m);
MapEntry<_DateTimeField, int> _day(int d) => MapEntry(_DateTimeField.day, d);
MapEntry<_DateTimeField, int> _hour(int h) => MapEntry(_DateTimeField.hour, h);
MapEntry<_DateTimeField, int> _minute(int m) =>
    MapEntry(_DateTimeField.minute, m);

DateTime _fromFields(Map<_DateTimeField, int> fields,
    {@required DateTime now}) {
  int _amPmOffset(int hour, int amPm) {
    if (amPm == 0 && hour == 12) return -12;
    if (amPm == 1 && hour >= 1 && hour <= 11) return 12;
    return 0;
  }

  final amPm = fields[_DateTimeField.ampm];
  final hour = fields[_DateTimeField.hour] == null
      ? 0
      : fields[_DateTimeField.hour] +
          _amPmOffset(fields[_DateTimeField.hour], amPm);

  return DateTime(
    fields[_DateTimeField.year] ?? DateTime.now().year,
    fields[_DateTimeField.month] ?? DateTime.january,
    fields[_DateTimeField.day] ?? 1,
    hour,
    fields[_DateTimeField.minute] ?? 0,
  );
}

This allows us formats like this:

    test('yMMMd jm', () {
      final format = DateFormat.yMMMd().add_jm();
      final dateParser = dateTimeFromFormat(format);
      expect(dateParser.parse('Oct 6, 1988 5:30 PM').value,
          DateTime(1988, 10, 6, 17, 30));
    });
``