Meeting Minutes for Week 2

[tonyshumlh]

Sprint Planning - 2024/05/06 Week 2

• We can start from user story for every week sprint
• We have define N user story, further break down user story into functions/work
• There are 3 layers of project planning
• Layer 0: (highest): the overview of the products’ functionalities and stories
• Layer 1 (higher): Epic, mainly linked to the user story(s) every week
• Layer 2 (lower): Task, daily tasks
• User Story 1: Proposal
• Task: Finish the proposal
  • Review the proposal by the team
  • Check with Simon on 1) the necessity of reproducible tool and 2) what does he mean by “proposed figures”, 3) if we should include the cost into Proposal Confirm with Simon the submission format of the proposal #20
  • Polish reference links Polish the repo's documents i.e. LICENSE/README/reports #23
  • Add the data example screencap into Session 3.3.1 Polish the repo's documents i.e. LICENSE/README/reports #23
  • Add the budget for the cost for development Create an estimation on the usage with LLM APIs #21
• User Story 2: User can get ML test completion score
  • User story for MVP Repo Analyzer #24
• User Story 3: Researcher/Leader get the checklist to push the best practice for ML test
  • User story for Checklist #25
• Task: Estimate the cost for development
  • e.g. API cost
  • Check with Simon 1) how to estimate the cost for development, 2) if Simon can lend his GPT4 and API key for our work in May 2024 Confirm with Simon the submission format of the proposal #20
  • Track the API usage and cost for Week 2 Provide an estimate on the OpenAI API usage #14
    Task: weekly meeting with Tiffany
  • confirm the meeting time with Tiffany Confirm the meeting date and time with Tiffany #22

[tonyshumlh]

Mentor Meeting 2024/05/09

• Use bib/common reference addition method in the checklist YAML
• Replace "Data Presence...." by "Topic: Data Presence...." in checklist YAML

"Data Presence" itself has no meaning, we should keep the meaningful thing as properties/keys, e.g. "Topic" and ensure consistent key 

• Make ppt 1) group name & date; 2) overview (content); 3) user stories; 4) checklist structure; 5) system graph; 6) prompt & POC; https://docs.google.com/presentation/d/1nP2x0-PkXpyT_Qtg18h5_dq5BqX9--Fi/edit#slide=id.g2d22ff621af_0_0
• Success metrics: can mention whether we can fulfill all user stories
• Confirm system structure, user stories with Tiffany in the first Partner meeting
• Move some words from proposal into Figure Caption
• Take reference of Neotoma Explorer Scenario (Simon paper) for user stories

[JohnShiuMK]

Partner Meeting Minutes - May 9, 2024

Key Points Discussed:

1. Test Cases on Data Quality and Handling: need to ensure the implementation of test cases for data quality and handling e.g. train-test split

   • relevant work: a paper (https://arxiv.org/pdf/2310.01402) by a professor from the University of Toronto
   • Tiffany will invite the professor to attend our final presentation.

2. Enhancing the Application:

   • Proposed the use of LLM to refine and adjust the "explanation" in our checklist for better user understanding
   • Suggested that the application should have the capability to build and retain knowledge over time, similar to how StoryBird.ai operates. This would allow the application to maintain some context and not start from scratch with each use.
     • Agreed that this feature is nice-to-have and should at least be discussed in the final report.
   • Discussed the potential for the application to tailor responses based on user-specific topic, which could be one of the topics in our checklist.

3. Meeting Coordination:

   • Agreed to confirm and align on the meeting time one day before the next meeting

[tonyshumlh]

Sprint Planning - 2024/05/13 Week 3

• TODO:
• User Story 2: User can get ML test completion score User story for MVP Repo Analyzer #24
  • John, Orix: Prototype the scripts for calling OpenAI API to feed prompt + code input and fetch response without hitting rate limit
  • John: Calling Code Analyzer script to breakdown codebase for code input
  • Tony: Prompt Engineering on the curated checklist to produce desired results via OpenAI API
  • Orix: Write a YAML parser to read the checklist items into a Python object
  • John, Tony: Write scripts to connect all items above to output Test Completeness Score
• User Story 3: Researcher/Leader get the checklist to push the best practice for ML test User story for Checklist #25
  • Orix, Yingzi: Add more items based on the research paper
• Presentation with Mentor and/or Partner
  • John: Demonstrate the checklist and check the quality and abundance of checklist
  • John: Showcase the workflow from the beginning till Test Completeness Score