Alice Engineering Comms 🪬 #1406

pdxjohnny · 2022-07-18T16:15:07Z

pdxjohnny
Jul 18, 2022
Maintainer

These are the engineering logs of entities working on Alice. If you
work on Alice please use this thread as a way to communicate to others
what you are working on. Each day has a log entry. Comment with your
thoughts, activities, planning, etc. related to the development of
Alice, our open source artificial general intelligence.

This thread is used as a communication mechanism for engineers so that
others can have full context around why entities did what they did
during their development process. This development lifecycle data helps
us understand more about why decisions were made when we re-read the
code in the future (via cross referencing commit dates with dates in
engineering logs). In this way we facilitate communication across
both time and space! Simply by writing things down. We live an an
asynchronous world. Let's communicate like it.

We are collaboratively documenting strategy and implementation as
living documentation to help community communicate amongst itself
and facilitate sync with potential users / other communities /
aligned workstreams.

Game plan
- Move fast, fix things. Live off the land. Failure is not an option.
References
- This thread in git grep -i friendly format: https://github.com/intel/dffml/tree/main/docs/discussions/alice_engineering_comms
- Video: General 5 minute intro to Alice
- https://github.com/intel/dffml/blob/main/docs/tutorials/rolling_alice/0000_forward.md
- https://github.com/intel/dffml/blob/main/docs/tutorials/rolling_alice/0000_preface.md
- https://gist.github.com/07b8c7b4a9e05579921aa3cc8aed4866
  - Progress Report Transcripts
- https://github.com/intel/dffml/tree/main/entities/alice/
- https://github.com/intel/dffml/tree/main/docs/tutorials/rolling_alice/0000_architecting_alice#what-is-alice
- 2nd Party #1401
  - docs: arch: Inventory #1207
  - docs: arch: 2nd and 3rd party plugins #1061
- service: sw: src: change: notify: Service to facilitate poly repo pull model dev tooling #1315
- docs: examples: innersource: Org health and issue prioritization #1287
- Aligned threads elsewhere (in order of appearance)
  - @dffml
    - DFFML Weekly Sync Meeting Minutes
      - Alice isn't mentioned here but the 2nd party work is, Alice will be our maintainer who helps us with the 2nd party ecosystem.
    - Alice #1369
  - @pdxjohnny
    - https://twitter.com/pdxjohnny/status/1522345950013845504
    - https://mastodon.social/@pdxjohnny/109320563491316354
- Google Drive: AliceIsHere
  - Supplementary resources, status update slide decks, cards (with instructions of how to get more printed), Entity Analysis Trinity drawio files, screenshots. Miscellaneous other files.
- async comms / asynchronous communications
  - https://twitter.com/SergioRocks/status/1579110239408095232

Engineering Log Process

Alice, every day at 7 AM in Portland's timezone create a system context (the tick)
- Merge with existing system context looked up from querying this thread if exists
  - In the future we will Alice will create and update system contexts.
    - We'll start with each day, then move to a week, then a fortnight, then 2 fortnights.
    - She'll parse the markdown document to rebuild the system context as if it's cached
      right before it would be synthesized to markdown, we then run updates and trigger
      update of the comment body. Eventually we won't use GitHub and just DID based stuff.
      We'll treat these all as trains of thought / chains of system contexts / state of the
      art fields.
      - Take a set of system contexts as training data
      - The system context which we visualize as a line dropped from the peak of a pyramid
        where it falls through the base.
      - We use cross domain conceptual mapping to align system contexts in a similar direction
        and drop ones which do are unhelpful, do not make the classification for "good"
      - What remains from our circular graph is a pyramid with the correct decisions
        (per prioritizer)
      - This line represents the "state of the art", the remembered (direct lookup) or
        predicted/inferred system contexts along this line are well rounded examples of
        where the field is headed, per upstream and overlay defined strategic plans
        and strategic principles
        
        References:
        
        $ git grep -C 5 -i principle -- docs/arch/
        
        Source: Alice #1369
- Inputs
  - date
    - Type: Union[str, Date]
    - Example: 2022-07-18
    - Default: did:oa:architype:current-date
- Add yesterdays unfinished TODO items to this train of though with the
- Create a document (docutils?)
  - Make the top level header date with "Notes" appended
  - Collect all previous days TODOs from within the individual entity comments within the thread for the days comment (the team summary for that day)
    - Drop any completed or struck through TODOs
    - Output a list item "TODO" with the underlying bullets with handle prepended and then the TODO contents
      - Create comments for individuals after this the current system context is posted and we have a live handle to it to reply with each individuals markdown document.
- Synthesis the document to markdown (there is a python lib out there that can do docutils to md, can't remember the name right now)
- Upsert comment in thread
Any time (it's a game)
- We find something we need to complete Alice #1369 that was just completed by someone else
  - Paste ![chaos-for-the-chaos-god](https://github.com/intel/dffml/assets/5950433/636969a1-1f0f-4c96-8812-f10fa403e79c)
  - chaos: diversity of authorship applied to clustering model run on all thoughts (dataflows) where each dataflow maps to line changed and the cluster classification is the author.
    - service: sw: src: change: notify: Service to facilitate poly repo pull model dev tooling #1315 (comment)
- We accelerate a train of thought
  - Paste 🛤️
- We use manifests or containers
  - Paste all the things meme ![oci-all-the-things](https://user-images.githubusercontent.com/5950433/222979759-0dd374b2-ee5f-4cbc-92d1-5cb8de078ee8.png)
- We find more ways to on-ramp data to the hypergraph
  - ![knowledge-graphs-for-the-knowledge-god](https://user-images.githubusercontent.com/5950433/222981558-0b50593a-c83f-4c6c-9aff-1b553403eac7.png)
- We do a live demo
  - ![live-demo-for-the-live-demo-god](https://user-images.githubusercontent.com/5950433/226699339-45b82b38-a7fc-4f2f-a858-e52ee5a6983d.png)
- We see alignment happening
  - ![such-alignment](https://user-images.githubusercontent.com/5950433/226707682-cfa8dbff-0908-4a34-8540-de729c62512f.png)
- We enable bisection or hermetic or cacheable builds
  - ![hash-values-everywhere](https://user-images.githubusercontent.com/5950433/230648803-c0765d60-bf9a-474a-b67e-4b4177dcb15c.png)

Source: unknown

pdxjohnny · 2022-07-18T16:28:51Z

pdxjohnny
Jul 18, 2022
Maintainer Author

2022-07-18 Engineering Logs

TODO
- @aliceoa, @pdxjohnny: Kick off OSS scans
  - Targeting collaboration with CRob on metrics insertion to OpenSSF DB
- @pdxjohnny: Finish Q3 plans (Gantt chart, meeting templates, etc.)
  - Generate template for auto creation to fill every meeting / fillable pre-meeting

1 reply

pdxjohnny Jul 18, 2022
Maintainer Author

2022-07-18 @pdxjohnny Engineering Logs

TODO
- Kick off OSS scans
  - Targeting collaboration with CRob on metrics insertion to OpenSSF DB
- Finish Q3 plans (Gantt chart, meeting templates, etc.)
  - Generate template for auto creation to fill every meeting / fillable pre-meeting
Future
- Engage with Loihi community
  - See what we can do here, not sure yet, play with system context / mitigation inference in devcloud?
  - https://www.intel.com/content/www/us/en/research/neuromorphic-community.html
  - https://download.intel.com/newsroom/2021/new-technologies/neuromorphic-computing-loihi-2-brief.pdf
  - https://www.intel.com/content/www/us/en/newsroom/news/intel-unveils-neuromorphic-loihi-2-lava-software.html
References
- https://medium.com/51nodes/decentralized-schema-registry-aa662b8db12b
- https://www.microsoft.com/security/blog/2021/10/06/microsofts-5-guiding-principles-for-decentralized-identities/
- https://ariadne.space/2022/07/17/how-efficient-can-cat1-be/
  - Usage of splice
- https://github.com/NVlabs/eg3d
  - Seeing from different perspectives, inter domain conceptual mapping, encoded sysctxs alternate mitigations
- https://github.com/robmarkcole/satellite-image-deep-learning
  - Knitting together system contexts (Alice could use for integration of various architectures)

pdxjohnny · 2022-07-19T15:38:19Z

pdxjohnny
Jul 19, 2022
Maintainer Author

2022-07-19 Engineering Logs

TODO
- @aliceoa, @pdxjohnny: Kick off OSS scans
  - Targeting collaboration with CRob on metrics insertion to OpenSSF DB
- @pdxjohnny: Finish Q3 plans (Gantt chart, meeting templates, etc.)
  - Generate template for auto creation to fill every meeting / fillable pre-meeting

1 reply

pdxjohnny Mar 7, 2023
Maintainer Author

2022-07-19 @pdxjohnny Engineering Logs

TODO
- Kick off OSS scans
  - Targeting collaboration with CRob on metrics insertion to OpenSSF DB
- Finish Q3 plans (Gantt chart, meeting templates, etc.)
  - Generate template for auto creation to fill every meeting / fillable pre-meeting
- Follow up with OneAPI folks
- Overlay to alice shouldi contribute to create git repos when found from forks of PyPi packages
  - Associated tutorial
    - Linked from README
- Finish out alice please contribute recommended community standards
  dynamic opimp for meta issue body creation
  - Associated tutorial
    - Linked from README and CONTRIBUTING
Some good spdx DAG stuff on how we turn source into build SBOM wise
- https://lists.spdx.org/g/Spdx-tech/message/4659
References
- https://github.com/nsmith5/rekor-sidekick
  - Rekor transparency log monitoring and alerting
  - Leverages Open Policy Agent
  - Found while looking at Open Policy Agent to see if we can serialize to JSON.
  - Possibly use to facilitate our downstream validation
    - service: sw: src: change: notify: Service to facilitate poly repo pull model dev tooling #1315
- https://mermaid-js.github.io/mermaid/#/c4c
  - Mermaid is working on native https://c4model.com support!
- W3C approves DIDs!

"Intel Corporation congratulates the DID Working Group on Decentralized Identifier (DID) 1.0 reaching W3C Recommendation status.

DID provides a framework to unify and consolidate multiple evolving identity systems. Consolidating identity systems within a single framework is useful for validating the authenticity of information and preserving its integrity as it is moved and processed among cloud, edge, and client systems. This potentially increases the capabilities of the Web to connect and unify multiple sources of information.

The continuing evolution of this work will be key to the development of new technologies in the fields of supply chain management and Internet of Things (IoT) devices and services. For example, a Birds of a Feather (BOF) discussion group at IETF Supply Chain Integrity, Transparency, and Trust (SCITT) has already highlighted DID as a useful approach in providing much needed structure for exchanging information through the supply chain, and the Web of Things (WoT) WG is planning to support DID for identifying and discovering IoT devices and metadata.

Intel Corporation supports this work and encourages the DID Working Group to continue working towards the convergence of widely implemented and adopted standardized best practices for identity in its next charter."

Eric Siow, Web Standards and Ecosystem Strategies Director, Intel Corporation

https://blog.devgenius.io/top-10-architecture-characteristics-non-functional-requirements-with-cheatsheat-7ad14bbb0a9b

For Vol 3, on mind control
- https://bigthink.com/the-present/sophists/

Unsent to Mike Scovetta: michael.scovetta (at) microsoft.com

Hi Mike,

Hope you’ve been well. It’s John from Intel. Thanks again to you and the team for welcoming me to the Identifying Security Threats working group meeting 2021-02-18 last year. We talked a bit about how Intel had a similar effort. I then changed roles hoping to get more involved with OpenSSF but then ended up getting told to be uninvolved. Now I switched roles again and involvement is in scope! Sorry for the lapse in communications.

I periodically check the minutes so I joined today and asked about the "Alpha-Omega" project from last week’s minutes which I then did some research on. We just started what looks to me to be an aligned project, coincidentally named Alice Omega Alpha: https://github.com/intel/dffml/tree/alice/entities/alice

Alice is the nickname for both the entity and the architecture, the Open Architecture, which is a methodology for interpretation of existing well established, formats, protocols, and other domain specific representations of architecture. What we end up with is some JSON, YAML, or other blob of structured data that we can use to build cross language tooling focused more on policy and intent, incorporating data from arbitrary sources to create a holistic picture of software across dependency boundaries by focusing on threat models.

Alice will be doing scans of open source projects and we’d still love to collaborate to contribute metrics to the OpenSSF metrics database, we can easily have her shoot applicable metrics off to that DB. We’ve also been looking at fusing VEX and DIDs to facilitate distributed vulnerability disclosure and patch distribution.

Unset to Jun Takei: jun.takei (at) intel.com

The W3C today issued the recommendation on DIDs. Jun I saw from Eric's
comment on the press release that the SCITT working group has an SCITT
Architecture which DID's might be suitable for.

The DFFML community is working on a project called Alice
https://github.com/intel/dffml/tree/alice/docs/tutorials/rolling_alice
she is intended to be a developer helper. She's also the way we data mine
source repositories (etc.).

She’s open source with a plugin system ("overlays") so we can write open source code
and then just add our internal integrations. This system relies on an abstraction of
architecture known as the Open Architecture. The Open Architecture, also known as
Alice, is a methodology for interpreting directed graphs of domain specific architectures.
Alice is the name we give both the entity and the architecture. We are hoping to
have Alice store and process information backed by directed graphs of DIDs, SBOMs, and
VEX info primarily. This sounds very similar to the SCITT Architecture. We would love to
collaborate with you both to help make SCITT a success. Alice is focused on analysis of
our software supply chain so as to ensure we conform to best practices. We would like
the analysis to serialize directly to an industry best practice format for that as well,
which SCITT looks to be.

To increase the level of trust in our supply chain we would like to ensure interoperability
up and down the stack. Ned is involved in the DICE space and communicated to me
that

Please let us know where things are at with your involvement with DIDs and SCITT so we
can be in sync with Intel's involvement and direction in this space. Please also let us know
how we could best establish an ongoing line of communication so as to build off and
contribute to where possible the work you're involved in.

References:

Unsent

To: Jun and Mike and Dan lorenc.d (at) gmail.com

I commented on the OpenSFF Stream 8 doc recommending that DIDs be looked at
as a way to exchange vulnerability information.

We've been looking potentially at a hybrid DID plus rekor
architecture (DIDs eventually as a proxy to)

References:

https://github.com/sigstore/rekor

pdxjohnny · 2022-07-20T14:49:37Z

pdxjohnny
Jul 20, 2022
Maintainer Author

2022-07-20 Engineering Logs

TODO
- @aliceoa, @pdxjohnny: Kick off OSS scans
  - Targeting collaboration with CRob on metrics insertion to OpenSSF DB
- @pdxjohnny: Finish Q3 plans (Gantt chart, meeting templates, etc.)
  - Generate template for auto creation to fill every meeting / fillable pre-meeting
- @dffml: Get involved in SCITT
  - Meetings
    - https://docs.google.com/document/d/1vf-EliXByhg5HZfgVbTqZhfaJFCmvMdQuZ4tC-Eq6wg/edit#
    - Weekly Monday at 8 AM Pacific
    - https://armltd.zoom.us/j/99133885299?pwd=b0w4aGorRkpjL3ZHa2NPSmRiNHpXUT09
  - Mailing list
    - https://www.ietf.org/mailman/listinfo/scitt
    - https://mailarchive.ietf.org/arch/browse/scitt/
  - Slack
    - https://mailarchive.ietf.org/arch/msg/scitt/PbvoKOX996cNHJEOrjReaNlum64/
    - Going to email Orie Steele orie (at) transmute.industries to ask for an invite.

2 replies

pdxjohnny Jul 20, 2022
Maintainer Author

2022-07-20 Identifying Security Threats WG

Mike leading
Marta
- Office hours
  - Place for Open Source maintainers to be able to ask community of security experts
  - Idea is to run first two sessions in August and September
  - Proposing two different timeslots to cover all geos
  - Question is will be be well staffed for both of those
  - She is collecting feedback right now on possibilities for those dates
  - Pinging folks who have shown interest in the past
  - What format?
    - People just show up and ask
    - Registration with topic they want to talk about
      - Allows us to prepare, consensus is currently we like this
      - Can grab right experts beforehand this way
  - Reaching out to logistics team for how we can communicate
    zoom links, etc.
  - Will test registration beginning of August
  - Will do doodle poll or something for slots
  - Jen is the one for the Zoom setup
  - Amir from ostif.org volunteering to answer questions
  - May want to do a blog in addition to twitter
  - Outreach maybe 4th or 5th, have the twitter points back
    to the blog to capture that.
Meeting time update
- We have been doing this at this time for about a year or so
- We previously alternated between two timeslots for Europe and Asia
- Should we keep this 10 AM Pacific timeslot?
  - Alternate between US and APAC friendly timezone
- Most other WGs are morning Pacific time
Technical Advisory Committee (TAC) update
- They are tasked with making sure we are delivering on our
  cohesive promise, part of that is visuabliity and transparency
  into the work that we do.
- We now have a formal reporting process
- It's not a periodic we're all invited to show up to the TAC meeting
  one slide per project.
  - What we're doing
  - Why we're doing it
- It's meant as an FYI, we are not asking for approval, we're letting them
  know what we're up to.
- Everyone who is driving a process or project or thing, please send Mike
  a single slide, what is it, why are we doing it, what the status is,
  what's coming next, and if you need anything
- Christine on metrics
- Luigi for SECURITY-INSIGHTS.yml`
- Mike will send out a template
  - Please fill and respond by Monday
- Mike says the metrics work should live under a working group, maybe this one, maybe best practices
- CRob might have an opinion here, as long as work gets done
- As an org OpenSSF would benefit by being less siloed
- Question on if we should align to streams?
- LFX specific definition of metrics in mobilization paper
- AR for Christine to sync with CRob and see what he thinks.
- Will raise with TAC next week.
A few action items for metrics from Christine
- Working groups are adopting streams from the mobilization plans
Mike: Alpha Omega
- A few people were on the public call earlier
- The recording will be on YouTube
- Mike will give the fast version of the presentation right now
- They are still hiring
- Exploring ways of allocating headcount other than direct hiring
- If you know anyone or are interested please apply or ping them!
- Alpha
  - Announced Node, Python, Eclipse
- Omega
  - Toolchain is pending
  - Waiting for legal approval due to the way the license for CodeQL works
  - Had a CVE in Node that got fixed earlier this month
  - RCE in JSHint that was bitrotted (unused) we removed
  - Two CVEs discloudsed yetserday and two more in the works (couple weeks to release ETA)
  - Found NodeJS vuln via systemcall tracing
    - It tires to query openssl.cnf and dumps strace logs to a repo
    - You then have a one stop show of show me every link package of when a binary starts, it does a DNS query
      - John: Sounds aligned with Alice's goals
- https://sos.dev coming under Alpha-Omega
  - Allows us to compensate dev directly
- How to participate
  - Improve security tools
  - https://sos.dev
  - Join working groups
  - Get on slack
Amir: Security Reviews
- Repo is looking good
- Updating with four new audits that ostif.org published last week
- At almost 100 reviews from Mike (Omega work), ostif.org, and community
- We're gaining traction, getting good stuff in there all the time
- Might need some help with the automated testing that get's done
  when we upload reviews.
- Feedback always welcome.
John: Collection of metric / Alpha-Omega data into shared DB
- https://github.com/intel/dffml/tree/alice/docs/tutorials/rolling_alice
- https://datatracker.ietf.org/doc/html/draft-birkholz-scitt-architecture
- https://www.w3.org/2022/07/pressrelease-did-rec.html.en
- https://docs.microsoft.com/en-us/azure/confidential-ledger/architecture
- Mike
  - Mike has been thinking about SCITT as a schema and rules on how one would assert facts, weither it's confidential compute or traditional permissions is impelmenetation details.
  - If metircs runs across you're repo and you have 30 contributors, great
  - As consumer, how can I discover that fact and trust that it's accruate
  - Could immaiget a world where things like Scorecard express the data as as SCITT assursion
  - You go and query that store and you say tell me everythig you know about foo and you get it all back
  - Until we have an implementation with WEb5 that's at at least beta, we could expore what that looks like.
    - John: We can do rekor for now, we'll bridge it all later target 1-2 years out
    - John: We have alignment. Time to execute. rekor + sigstore for metric data atteststation signed with github odic tokens. We care about data provenance. We will later bridge into web5 space used as central points of comms given DID as effectively the URL or the future. This is in realtion to what we talked to Melvin about with data provenance. We need to start planning how we are going to build up this space now so we can have provenance on thoughts later. This provenance could be for example on inference derived from provenance from training data and model training env and config. This will allow us to ensure the prioritizer make decisions based on Sprit of the law / aka intent based policy derived from Trinity of Static Analysis, Dynamic Analysis, and Human Intent.
      - Living Threat Model threats, mitigations, trust boundaries as initial data set for cross domain conceptual mapping of the the trinity to build pyramid of thought alignment to strategic principles.
      - One of our strategic plans / principles says: "We must be able to trust the sources of all input data used for all model training was done from research studies with these ethical certifications"
        
        This allows us to write policies (Open Policy Agent to JSON to DID/VC/SCITT translation/application exploration still in progress) for the organizations we form and apply them as overlays to flows we execute where context appropriate. These overlaid flows define the trusted parties within that context as applicable to the active organizational policies as applicable to the top level system context.
        
        The policy associated with the principle that consumes the overlaid trust attestations we will implement and LTM auditor for which checks the SCITT provenance information associated with the operation implementations and the operation implementation network, input network, etc. within the orchestrators trust boundary (TODO need to track usages / reuse of contexts ictx, nctx, etc. with something predeclared, aka at runtime if your Operation data structure doesn't allowlist your usage of it you can pass it to a subflow for reuse. This allows us to use the format within our orchrestration and for static analysis because we can use this same format to describe the trust boundry proeprties that other domain sepcific represenatations of architecture have, for instance we could if we were doing and Open Architecture (OA) Intermediate Representation (IR) for and ELF file we might note that the input network context is not reused from the top level system context. Where as if we did an OA IR for Python code we would say that the input network is reused from the top level system context (it has access to that memory region, whereas when you launch and ELF you look access to the parents memory region, typically).
  - Christine
    - Looking at trying to connect all the different data sources
References
TODO
- @pdxjohnny
  - Reach out to Christine about metrics collaboration
  - Respond with slides for Mike if he asks

pdxjohnny · 2022-07-21T22:04:10Z

pdxjohnny
Jul 21, 2022
Maintainer Author

2022-07-21 Engineering Logs

https://docs.rs/differential-dataflow/latest/differential_dataflow/
https://lists.spdx.org/g/Spdx-tech/message/4673
- It is not just a matter of your software, it is a fundamental design question whether to maintain separation between the logical model and its serializations. Maintaining separation shouldn't be a matter of personal preference, it's good software engineering. The OWL Web Ontology Language https://www.w3.org/TR/owl2-overview/ has an excellent diagram illustrating the separation between semantics and syntax. Several serializations are defined in OWL (Manchester Syntax, Functional Syntax, RDF/XML, OWL/XML, and Turtle), and more syntaxes have been added since (JSON-LD, RDF-star, ...).

0 replies

pdxjohnny · 2022-07-23T17:54:28Z

pdxjohnny
Jul 23, 2022
Maintainer Author

2022-07-23

https://blog.ciaranmcnulty.com/2022-05-12-multiple-build-contexts

0 replies

pdxjohnny · 2022-07-24T15:54:29Z

pdxjohnny
Jul 24, 2022
Maintainer Author

2022-07-28 Alice Intelligence/Open Architecture Working Group Initial Meeting

Meeting info
- 8-9 AM Pacific
- https://meet.google.com/kox-ssqn-kjd

0 replies

pdxjohnny · 2022-07-25T14:50:20Z

pdxjohnny
Jul 25, 2022
Maintainer Author

2022-07-25 Engineering Logs

TODO
- @aliceoa, @pdxjohnny: Kick off OSS scans
  - Targeting collaboration with CRob on metrics insertion to OpenSSF DB
- @pdxjohnny: Finish Q3 plans (Gantt chart, meeting templates, etc.)
  - Generate template for auto creation to fill every meeting / fillable pre-meeting
- @dffml: Get involved in SCITT
  - Meetings
    - https://docs.google.com/document/d/1vf-EliXByhg5HZfgVbTqZhfaJFCmvMdQuZ4tC-Eq6wg/edit#
    - Weekly Monday at 8 AM Pacific
    - https://armltd.zoom.us/j/99133885299?pwd=b0w4aGorRkpjL3ZHa2NPSmRiNHpXUT09
  - Mailing list
    - https://www.ietf.org/mailman/listinfo/scitt
    - https://mailarchive.ietf.org/arch/browse/scitt/
  - Slack
    - https://mailarchive.ietf.org/arch/msg/scitt/PbvoKOX996cNHJEOrjReaNlum64/
    - Going to email Orie Steele orie (at) transmute.industries to ask for an invite.

2 replies

pdxjohnny Jul 25, 2022
Maintainer Author

2022-07-25 Supply Chain Integrity, Transparency and Trust (SCITT)

TODO
- Get involved in SCITT
  - Meetings
    - https://docs.google.com/document/d/1vf-EliXByhg5HZfgVbTqZhfaJFCmvMdQuZ4tC-Eq6wg/edit#
    - Weekly Monday at 8 AM Pacific
      - Joining today
    - https://armltd.zoom.us/j/99133885299?pwd=b0w4aGorRkpjL3ZHa2NPSmRiNHpXUT09
  - Mailing list
    - https://www.ietf.org/mailman/listinfo/scitt
    - https://mailarchive.ietf.org/arch/browse/scitt/
  - Slack
    - https://mailarchive.ietf.org/arch/msg/scitt/PbvoKOX996cNHJEOrjReaNlum64/
    - Going to email Orie Steele orie (at) transmute.industries to ask for an invite.
Links
Folks will be in Philly on Thursday for meeting
- There is a remote link (see mailing list?) for the Thursday meeting
Typical global warming centric chit chat
Others at RATs meeting or busy with other IETF activities
Introductions
- Kelvin Cusack
  - Filling in for John from his org
- John Andersen
  - Connecting dots between this and OpenSSF
    - Yogesh was excited to see someone from Intel
      - Intel is involved in RATs but not as much here
- Kiran Karunakaran
  - Microsoft
  - On Kay Williams's team
  - Will likely lead this meeting in the future
Upcoming Birds of Feather (BoF)
- You need to register here: https://docs.google.com/document/d/1vf-EliXByhg5HZfgVbTqZhfaJFCmvMdQuZ4tC-Eq6wg/edit?pli=1#heading=h.214jg0n2xjhp
  - There is registration for remote
- Problem Statement currently scoped around software
- We went back to a more scoped problem statement is that we want to form a formal working group in the IETF for SCITT.
- In order to form it we have to have stuffiest people show interest in the problem space
- Need problem space and charter so that it's scoped enough that the leadership is confident that the group can make progress
- The early design proposal for the SCITT transparency service is that the service is content agnostic
- any kind of metadata could be provided and retrieved
- SBOMs, Software Test Cases, Hardware BOMs, Test results on other types of products
- Electronic ballots (Ray, see mailing list), oil, gas, physical goods (U.S. govt.)
- In order to gain confidence from the leadership at IETF to form the WG we felt it was critical to narrow the scope for now to software
  - Leadership thought scope was too big at first
Thoughts around scope
- Charter is focused on software, an attainable goal
- Once we have a WG we can later broaden the scope via re-chartering
We will design something that works for hardware and for software
- We are hanging software window curtains but we are looking at everything
Software systems interact with everything else
- Dick Brooks (REA), any manifest could be signed and processes with SCITT
- It's just metadata, what was issued, how it was issued
  - @dffml: We are encoding system contexts, Alice, into the chain, one place where she will live.
Opens
- Open Policy Agent (mentioned in meeting minutes doc future topics)
  - What are your plans / thoughts around Open Policy Agent and encoding policies into SCITT?
    - Policy can be used in two places
      - Policy for what can be put onto register
        
        Some registries might constrain themselves for what types of data they allow
    - Policy for someone evaluating the contents of the registry to make discussions for fitness of use
  - REGO also considered as a policy language
    - Perhaps decide on multi
    - This policy discussion will happen in this WG for now, then maybe a sub working group
- Dick Brooks: Mentions HR4081
  - On topic for what we are
  - Talked about attestations for devices containing a camera and microphone and is connected to the internet
    - There will need to be an attestation from the device
      - Dick submitted to local rep to include software attestations as well
    - https://www.congress.gov/bill/117th-congress/house-bill/4081
      https://www.congress.gov/bill/117th-congress/house-bill/4081/text
Producers and other parties provide content into the system, attestations, claims, recorded into SCITT ledger
Dick contacted his congress person to ask to add an amendment to HR4081
- Amendment for smartphone apps to provide a trust score
  - Tie in with OpenSSF metrics database to grab the security of repos involved
- Dick
  - Proposed amendment I mentioned for HR 4081:
  - "Require smart phone app stores to include a software supply chain trust score for each app". This gives consumers the ability to check trustworthiness before installing an app,
Ray: think about attestations is different than the transparency ledger
- Thinks it's a lot to bite off to do both
- Are there IoT folks that might have more attestation experience we could tap into?
- Is there a sub-working group focused on device attestations (in response to HR4081)
  - Device attestations could be recorded tin the transparency ledger
  - TCG DICE WG is a target point of engagement (UCAN, CBOR, DID).
    - https://trustedcomputinggroup.org/work-groups/dice-architectures/
    - https://www.trustedcomputinggroup.org/wp-content/uploads/Device-Identifier-Composition-Engine-Rev69_Public-Review.pdf
    - Looking at hardware actively attesting
    - Microsoft has an Open Source implementation on GitHub
  - This attestation stuff starts to look at real life commerce, Ray thinks it's important to
Joshua Lock
- On software attestations, I have been working on a page for the SLSA website to describe the model we're working with. I can share to the SCITT list once the change is merged.
- IIRC the SCITT draft standards refer to a 'software attestation" as a "claim", to disambiguate from RATS & TCG attestations
Remote Attestation and Device Attestation
- Embraced COSE and CBOR
  - Also in SCITT
  - Hopefully we converge on underlying formats for both in-toto style and remote attestation style attestations
There are also NIST attestations
Vuln information mentioned by Kay as possible content inserted into SCITT
- This is a goal of ours with our CVE Binary Tool engagement
- We also could encode SBOMs from the systems that built them, we could patch sigstore to insert into a SCITT ledger

pdxjohnny · 2022-07-26T13:09:49Z

pdxjohnny
Jul 26, 2022
Maintainer Author

2022-07-26 Engineering Logs

TODO
- @aliceoa, @pdxjohnny: Kick off OSS scans
  - Targeting collaboration with CRob on metrics insertion to OpenSSF DB
- @pdxjohnny: Finish Q3 plans (Gantt chart, meeting templates, etc.)
  - Generate template for auto creation to fill every meeting / fillable pre-meeting
- @dffml: Get involved in SCITT
  - Meetings
    - https://docs.google.com/document/d/1vf-EliXByhg5HZfgVbTqZhfaJFCmvMdQuZ4tC-Eq6wg/edit#
    - Weekly Monday at 8 AM Pacific
    - https://armltd.zoom.us/j/99133885299?pwd=b0w4aGorRkpjL3ZHa2NPSmRiNHpXUT09
  - Mailing list
    - https://www.ietf.org/mailman/listinfo/scitt
    - https://mailarchive.ietf.org/arch/browse/scitt/
  - Slack
    - https://mailarchive.ietf.org/arch/msg/scitt/PbvoKOX996cNHJEOrjReaNlum64/
    - Going to email Orie Steele orie (at) transmute.industries to ask for an invite.

1 reply

pdxjohnny · 2022-07-27T20:43:28Z

pdxjohnny
Jul 27, 2022
Maintainer Author

2022-07-27 Engineering Logs

References
- kaniko coder k3d digitalocean
  - The following were issues with kind which might also effect us
    - Multi-stage builds fail when run in kind GoogleContainerTools/kaniko#2164
    - tektoncd/pipeline@6542823
  - It doesn't look like kaniko currently supports multi context builds
- Great example of communication and meeting procedures link to code
  - https://lists.spdx.org/g/Spdx-tech/message/4699

1 reply

pdxjohnny Jul 28, 2022
Maintainer Author

2022-07-27 @pdxjohnny Engineering Logs

TODO
- Get involved in SCITT
  - Meetings
    - https://docs.google.com/document/d/1vf-EliXByhg5HZfgVbTqZhfaJFCmvMdQuZ4tC-Eq6wg/edit#
    - Weekly Monday at 8 AM Pacific
      - Joining today
    - https://armltd.zoom.us/j/99133885299?pwd=b0w4aGorRkpjL3ZHa2NPSmRiNHpXUT09
  - Mailing list
    - https://www.ietf.org/mailman/listinfo/scitt
    - https://mailarchive.ietf.org/arch/browse/scitt/
  - Slack
    - https://mailarchive.ietf.org/arch/msg/scitt/PbvoKOX996cNHJEOrjReaNlum64/
    - Going to email Orie Steele orie (at) transmute.industries to ask for an invite.
- Kick off OSS scans
  - Targeting collaboration with CRob on metrics insertion to OpenSSF DB
- Finish Q3 plans (Gantt chart, meeting templates, etc.)
  - Generate template for auto creation to fill every meeting / fillable pre-meeting
- Follow up with OneAPI folks
- Overlay to alice shouldi contribute to create git repos when found from forks of PyPi packages
  - Associated tutorial
    - Linked from README
- Finish out alice please contribute recommended community standards
  dynamic opimp for meta issue body creation
  - Associated tutorial
    - Linked from README and CONTRIBUTING
- Software Analysis Trinity diagram showing Human Intent, Static Analysis, and Dynamic Analysis to represent the soul of the software / entity and the process taken to improve it.
  - SoftwareAnalysisTrinity.drawio.xml

Refactoring and Thinking About Locking of Repos for Contributions

Metadata
- Date: 2022-07-27 20:30 UTC -7
Saving this diff which was some work on dynamic application of overlay
so as to support fixup of the OpImp for meta_issue_body()'s inputs.
- We are going to table this for now for time reasons, but if someone
  wants to pick it up before @pdxjohnny is back in September, please
  give it a go (create an issue).
Noticed that we have an issue with adding new files and locking. The current
lock is on the git_repository/GitRepoSpec.
- We then convert to AliceGitRepo, at which point anything take AliceGitRepo
alice
- Goal: Display Alice and software analysis trinity
- https://free-images.com/search/?q=alice%27s+adventures+in+wonderland&cat=st
- https://free-images.com/display/de_alices_abenteuer_im_43.html
- https://github.com/KhorSL/ASCII-ART
- Completed in d067273

$ python ascii_art.py /mnt/c/Users/Johnny/Downloads/alice-looking-up-white-background.png

$ alice
usage: alice [-h] [-log LOG] {please,shouldi,threats} ...

                            .,*&&888@@#&:,
                          .:&::,...,:&#@@@#:.
                         .o,.       ..:8@@#@@+
                        .8o+,+o*+*+,+:&#@@#8@@.
                        &8&###@#&..*:8#@@#@#@@&+.
                       ,@:#@##@@8,:&#@@@###@88@@.
                      ,#@8&#@@@#o:#@@@@#8#@#8+&#.
                     +8####@@@@###@@@888#@@@#oo#.
                   .*8@###@@@@@@@@@#o*#@@#@@#8o@,
                  +###@#o8&#@@##8::##@@@&&#@8#&+
                  o@8&#&##::.,o&+88#&8##8*@@#@#,
                 .##888&&oo#&o8###8&o##8##&####8,
                .&#@8&:+o+&@@@#8#&8:8@@@@@#8@@@oo+
               ,&&#@##oo+*:@###&#88,@@@@#@o&##&8#@o,.
              ,#&###@@8:*,#o&@@@@##:&#@###*.&o++o#@@#&+
              o8&8o8@#8+,,#.88#@#&@&&#@##++*&#o&&&#@@@@.
              *88:,#8&#,o+:+@&8#:8@8&8#@@&o++,*++*+:#@@*.
              .+#:o###@8o&8*@o&o8@o888@@@o+:o*&&,@#:&@@@,
                *+&@8&#@o#8+8*#+8#+88@@@@@@&@###8##@8:*,
                  +o.@##@@@&88@*8@:8@@@@@@:.. ,8@:++.
                    +&++8@@@@##@@@@@@@@@@@+    88
                    &.   *@8@:+##o&888#@@@,   .#+
                    &.   ,@+o,.::+*+*:&#&,    ,@.
                    &.   .@8*,. ,*+++.+*     :8+
                    :+   .#@::. .8:.:**    .8@@o,
                    .o.   #@+   :@,.&*   .:@@@@@@8**.
                     +&.  :@o,+.*o,*,  .*@@@@@@@@@@#o
                   .*:&o.  8@o:,*:,  .o@@#8&&@@@@#@@@*
                 ,*:+:::o.*&8+,++  ,&@@#:  * :@@88@@@#:.
               ,::**:o:.,&*+*8:  *8@@##o   *,.8@@#8#@#@#+
              *:+*&o8:. ,o,o:8@+o@@88:*@+  +: +#@#####8##&.
            ,:&::88&,   .&:#o#@@@#,+&&*#&. .:,.&#@#88#####&,
           +::o+&8:.    :##88@@@@:.:8o+&8&. .. +8###&8&##&88*
         .:*+*.8#:    ,o*.+&@@#@8,,o8*+8##+    .+#8##8&&#8888:.
        ,:o., &#8.  .:8*.  .o, &#,*:8:+,&*:,    .8@@#o&&##8:&#8.
      .*o.*,+o8#*  +8&,  .::. .88.+:8o: ,+:,    ,o#@#8&o8##&#8+
     +o, .+,,o#8+,8@o**.,o*,  :8o +*8#*  +&,    ,*o@@#@&8&oo8&:,
    oo*+,,,*8@#..&@8:**:oo+. +8#* *+#@:...oo+  .**:8@@@ooo&:&o##+
    ::+..,++#@,.:##o&o**,....oo#++#8#@:.,:8&:.....*&@@#:oo*&oo&#@*
     .+**:*8@o,+##&o:+,,,+,,o*8#,,8@#@:,,+*o*++,,,,+&#@8*8o88&::*.        .,,,,,++,
         ..8@++#@#88:,,,.,,,:+#&,,#@@#:,,.,&o*,.+++*:#@8+:*+.    ......,:+*&,,.....
            +:&8#@@##8&+,,,***@&,.8@@@*,,,.:o8&o&*o&o&o.     .,.****::*:o*:o*o+,.
                 ...,*:*o&&o*8@@&o8@@@8+,,+:&&:+,...     ,++*&oo&8&&&oo#@##8#&8:.
                        o@#@@@@#@@@@@@@,.....  ..,,.+*::o#@##@##@#@#########@@@8:,.
                        ,@##@@88#@@@@@8     .:***oo*#8###8#@#@#@#@####@#@###@@#8&#:
                         8+.,8+..,*o#@+     ,o+o88&88###@8#######@8#8#88#8#88##88#&
                         *o  *+     #8       .  ,*o&#@##@@@@@@@@@######8#888&&oo:8:
                          8, ,&    +@*        .ooo&#@@@@@#@@@@@@####@##8#8##oo:o&:,
                          +&  &,  .@#.        .:8#@@@@@@@@@@##8#####8#o&&#8*:8&&8:
                           o* ,o  o@&        +o#@@@@@@@@#o&o88:&+ooo&:*::o:o&**o.:*+
                           .8. 8.,o#8    .+&#@@@@@@@@&o+,::*+*:+:, ,. ,.. .,. ,.
                            8. 8.,.&@:*:&@@@@@@@@8o+,    ,.
                           :@o:#,,o8&:o&@@@@#&:+.
                          .@@@@@@@@@@@#8&o+,
                           ,*:&#@#&o*,..

                                /\
                               /  \
                              Intent
                             /      \
                            /        \
                           /          \
                          /            \
                         /              \
                        /  Alice is Here \
                       /                  \
                      /                    \
                     /______________________\

             Dynamic Analysis          Static Analysis

    Alice's source code: https://github.com/intel/dffml/tree/alice/entities/alice
    How we built Alice: https://github.com/intel/dffml/tree/alice/docs/tutorials/rolling_alice
    How to extend Alice: https://github.com/intel/dffml/blob/alice/entities/alice/CONTRIBUTING.rst
    Comment to get involved: https://github.com/intel/dffml/discussions/1406


positional arguments:
  {please,shouldi,threats}

optional arguments:
  -h, --help            show this help message and exit
  -log LOG              Logging Level

TODO
- Auto fork repo before push
  - Update origin to push to
  - Create PR
- Update README to fix demos
- Update CONTRIBUTING with tutorial on adding
  CONTRIBUTING.md check and contribution

entities/alice/alice/timelines.py

"""
Helpers for the timelines we support
"""

# Trinity Day 0
ALICE_DAY_0_GREGORIAN = datetime.datetime(2022, 4, 16)

def date_alice_from_gregorian(date: str) -> int:
    # TODO
    return ALICE_DAY_0_GREGORIAN

diff --git a/dffml/base.py b/dffml/base.py
index fea0ef7220..9d6cd886fa 100644
--- a/dffml/base.py
+++ b/dffml/base.py
@@ -237,6 +237,7 @@ def convert_value(arg, value, *, dataclass=None):
                         # before checking if the value is an instance of that
                         # type. Since it doesn't make sense to check if the
                         # value is an instance of something that's not a type.
+                        print(possible_type, value)
                         if isinstance(possible_type, type) and isinstance(
                             value, possible_type
                         ):
diff --git a/dffml/df/system_context/system_context.py b/dffml/df/system_context/system_context.py
index e055a343f1..063547ad0c 100644
--- a/dffml/df/system_context/system_context.py
+++ b/dffml/df/system_context/system_context.py
@@ -90,11 +90,11 @@ class SystemContextConfig:
     # links: 'SystemContextConfig'
     overlay: Union["SystemContextConfig", DataFlow] = field(
         "The overlay we will apply with any overlays to merge within it (see default overlay usage docs)",
-        default=APPLY_INSTALLED_OVERLAYS,
+        default=None,
     )
     orchestrator: Union["SystemContextConfig", BaseOrchestrator] = field(
         "The system context who's default flow will be used to produce an orchestrator which will be used to execute this system context including application of overlays",
-        default_factory=lambda: MemoryOrchestrator,
+        default=None,
     )
 
 
@@ -131,6 +131,7 @@ class SystemContext(BaseDataFlowFacilitatorObject):
         )
         # TODO(alice) Apply overlay
         if self.config.overlay not in (None, APPLY_INSTALLED_OVERLAYS):
+            print(self.config.overlay)
             breakpoint()
             raise NotImplementedError(
                 "Application of overlays within SystemContext class entry not yet supported"
diff --git a/dffml/high_level/dataflow.py b/dffml/high_level/dataflow.py
index d180b5c302..d595ae1cb4 100644
--- a/dffml/high_level/dataflow.py
+++ b/dffml/high_level/dataflow.py
@@ -206,12 +206,25 @@ async def run(
             # the of the one that got passed in and the overlay.
             if inspect.isclass(overlay):
                 overlay = overlay()
+            # TODO Move this into Overlay.load. Create a system context to
+            # execute the overlay if it is not already.
+            known_overlay_types = (DataFlow, SystemContext)
+            if not isinstance(overlay, known_overlay_types):
+                raise NotImplementedError(f"{overlay} is not a known type {known_overlay_types}")
+            if isinstance(overlay, DataFlow):
+                overlay = SystemContext(
+                    upstream=overlay,
+                )
             # TODO(alice) overlay.deployment("native.python.overlay.apply")
             apply_overlay = overlay.deployment()
             async for _ctx, result in apply_overlay(
                 dataflow=dataflow,
             ):
+                print("FEEDFACE", _ctx, result)
+                breakpoint()
+                return
                 continue
+
                 # TODO
                 resultant_system_context = SystemContext(
                     upstream=result["overlays_merged"], overlay=None,
diff --git a/dffml/overlay/overlay.py b/dffml/overlay/overlay.py
index 13a50d9c10..0a01d38de9 100644
--- a/dffml/overlay/overlay.py
+++ b/dffml/overlay/overlay.py
@@ -124,7 +124,7 @@ DFFML_MAIN_PACKAGE_OVERLAY = DataFlow(
             stage=Stage.OUTPUT,
             inputs={
                 "merged": DataFlowAfterOverlaysMerged,
-                "dataflow_we_are_applying_overlays_to_by_running_overlay_dataflow_and_passing_as_an_input": DataFlowWeAreApplyingOverlaysToByRunningOverlayDataflowAndPassingAsAnInput,
+                "upstream": DataFlowWeAreApplyingOverlaysToByRunningOverlayDataflowAndPassingAsAnInput,
             },
             outputs={"overlayed": DataFlowAfterOverlaysApplied,},
             multi_output=False,
@@ -208,15 +208,12 @@ merge_implementations(
 DFFML_OVERLAYS_INSTALLED.update(auto_flow=True)
 
 # Create Class for calling operations within the System Context as methods
-DFFMLOverlaysInstalled = SystemContext.subclass(
-    "DFFMLOverlaysInstalled",
-    {
-        "upstream": {"default_factory": lambda: DFFML_OVERLAYS_INSTALLED},
-        # TODO(alice) We'll need to make sure we have code to instantiate and
-        # instance of a class if only a class is given an not an instance.
-        "overlay": {"default_factory": lambda: None},
-        "orchestrator": {"default_factory": lambda: MemoryOrchestrator()},
-    },
+DFFMLOverlaysInstalled = SystemContext(
+    upstream=DFFML_OVERLAYS_INSTALLED,
+    # TODO(alice) We'll need to make sure we have code to instantiate and
+    # instance of a class if only a class is given an not an instance.
+    overlay=None,
+    orchestrator=MemoryOrchestrator(),
 )
 
 # Callee
diff --git a/entities/alice/alice/please/contribute/recommended_community_standards/alice/operations/github/issue.py b/entities/alice/alice/please/contribute/recommended_community_standards/alice/operations/github/issue.py
index 46d20c8c85..fff5d4928b 100644
--- a/entities/alice/alice/please/contribute/recommended_community_standards/alice/operations/github/issue.py
+++ b/entities/alice/alice/please/contribute/recommended_community_standards/alice/operations/github/issue.py
@@ -18,6 +18,14 @@ from ....recommended_community_standards import AliceGitRepo, AlicePleaseContrib
 from ....dffml.operations.git.contribute import AlicePleaseContributeRecommendedCommunityStandardsOverlayGit
 
 
+GitHubIssue = NewType("GitHubIssue", str)
+
+
+@dataclasses.dataclass
+class RecommendedCommunityStandardContribution:
+    path: pathlib.Path
+    issue: GitHubIssue
+
 
 class AlicePleaseContributeRecommendedCommunityStandardsOverlayGitHubIssue:
     """
@@ -39,6 +47,7 @@ class AlicePleaseContributeRecommendedCommunityStandardsOverlayGitHubIssue:
     MetaIssueTitle = NewType("MetaIssueTitle", str)
     MetaIssueBody = NewType("MetaIssueBody", str)
 
+    # TODO This should only be run if there is a need for a README
     # body: Optional['ContributingIssueBody'] = "References:\n- https://docs.github.com/articles/setting-guidelines-for-repository-contributors/",
     async def readme_issue(
         self,
@@ -79,13 +88,40 @@ class AlicePleaseContributeRecommendedCommunityStandardsOverlayGitHubIssue:
             """
         ).lstrip()
 
-    # TODO(alice) There is a bug with Optional which can be revield by use here
+
     @staticmethod
+    async def readme_contribution(
+        issue: "ReadmeIssue",
+        path: AlicePleaseContributeRecommendedCommunityStandards.ReadmePath,
+    ) -> RecommendedCommunityStandardContribution:
+        return RecommendedCommunityStandardContribution(
+            path=path,
+            issue=issue,
+        )
+
+
+    """
+    @dffml.op(
+        stage=dffml.Stage.OUTPUT,
+    )
+    async def collect_recommended_community_standard_contributions(
+        self,
+    ) -> List[RecommendedCommunityStandardContribution]:
+        async with self.octx.ictx.definitions(self.ctx) as od:
+            return [item async for item in od.inputs(RecommendedCommunityStandardContribution)]
+    """
+
+
+    # TODO(alice) There is a bug with Optional which can be revield by use here
     def meta_issue_body(
         repo: AliceGitRepo,
         base: AlicePleaseContributeRecommendedCommunityStandardsOverlayGit.BaseBranch,
-        readme_path: AlicePleaseContributeRecommendedCommunityStandards.ReadmePath,
-        readme_issue: ReadmeIssue,
+        # recommended_community_standard_contributions: List[RecommendedCommunityStandardContribution],
+        # TODO On @op inspect paramter if Collect is found on an input, wrap the
+        # operation in a subflow and add a generic version of
+        # collect_recommended_community_standard_contributions to the flow as an
+        # autostart or triggered via auto start operation.
+        # recommended_community_standard_contributions: Collect[List[RecommendedCommunityStandardContribution]],
     ) -> "MetaIssueBody":
         """
         >>> AlicePleaseContributeRecommendedCommunityStandardsGitHubIssueOverlay.meta_issue_body(
@@ -98,6 +134,7 @@ class AlicePleaseContributeRecommendedCommunityStandardsOverlayGitHubIssue:
         - [] [License](https://github.com/intel/dffml/blob/main/LICENSE)
         - [] Security
         """
+        readme_issue, readme_path = recommended_community_standard_contributions[0]
         return "\n".join(
             [
                 "- ["

pdxjohnny · 2022-07-28T10:28:59Z

pdxjohnny
Jul 28, 2022
Maintainer Author

2022-07-28 Engineering Logs

1 reply

pdxjohnny Jul 28, 2022
Maintainer Author

2022-07-28 @pdxjohnny Engineering Logs

TODO
- Get involved in SCITT
  - Meetings
    - https://docs.google.com/document/d/1vf-EliXByhg5HZfgVbTqZhfaJFCmvMdQuZ4tC-Eq6wg/edit#
    - Weekly Monday at 8 AM Pacific
      - Joining today
    - https://armltd.zoom.us/j/99133885299?pwd=b0w4aGorRkpjL3ZHa2NPSmRiNHpXUT09
  - Mailing list
    - https://www.ietf.org/mailman/listinfo/scitt
    - https://mailarchive.ietf.org/arch/browse/scitt/
  - Slack
    - https://mailarchive.ietf.org/arch/msg/scitt/PbvoKOX996cNHJEOrjReaNlum64/
    - Going to email Orie Steele orie (at) transmute.industries to ask for an invite.
- Kick off OSS scans
  - Targeting collaboration with CRob on metrics insertion to OpenSSF DB
- Finish Q3 plans (Gantt chart, meeting templates, etc.)
  - Generate template for auto creation to fill every meeting / fillable pre-meeting
- Overlay to alice shouldi contribute to create git repos when found from forks of PyPi packages
  - Associated tutorial
    - Linked from README
- Finish out alice please contribute recommended community standards
  dynamic opimp for meta issue body creation
  - Associated tutorial
    - Linked from README and CONTRIBUTING
- Software Analysis Trinity diagram showing Human Intent, Static Analysis, and Dynamic Analysis to represent the soul of the software / entity and the process taken to improve it.
  - SoftwareAnalysisTrinity.drawio.xml
Noticed that we have an issue with adding new files and locking. The current
lock is on the git_repository/GitRepoSpec.
- We then convert to AliceGitRepo, at which point anything take AliceGitRepo
- alice: please: contribute: recommended community standards: Refactoring into overlays associated with each file contributed
  - Completed in 1a71dbe

43

(Pdb) custom_run_dataflow_ctx.config.dataflow.seed.append(dffml.Input(value=repo, definition=definition, origin=('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme', 'result')))
(Pdb) custom_run_dataflow_ctx.config.dataflow.seed
[Input(value=origin, definition=writable.github.remote.origin), Input(value=master, definition=repo.git.base.branch), Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-hxnacg5_', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo)]

Attempting to figure out why an operation is not being called
- contribute_readme_md should be getting base, but is not.

{'_': {ReadmeGitRepo: [Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo)],
       ReadmeIssue: [Input(value=https://github.com/pdxjohnny/testaaaa/issues/108, definition=ReadmeIssue)],
       ReadmePath: [Input(value=/tmp/dffml-feature-git-68ghk7vd/README.md, definition=ReadmePath)],
       github.pr.body: [Input(value=Closes: https://github.com/pdxjohnny/testaaaa/issues/108, definition=github.pr.body)],
       repo.git.base.branch: [Input(value=master, definition=repo.git.base.branch)],
       repo.readme.git.commit.message: [Input(value=Recommended Community Standard: README

Closes: https://github.com/pdxjohnny/testaaaa/issues/108
, definition=repo.readme.git.commit.message)],
       writable.github.remote.origin: [Input(value=origin, definition=writable.github.remote.origin)]},
 'alternate_definitions': [],
 'by_origin': {('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:determin_base_branch', 'result'): [Input(value=master, definition=repo.git.base.branch)],
               ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGitHub:github_owns_remote', 'result'): [Input(value=origin, definition=writable.github.remote.origin)],
               ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme', 'result'): [Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo)],
               ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:create_readme_file_if_not_exists', 'result'): [Input(value=/tmp/dffml-feature-git-68ghk7vd/README.md, definition=ReadmePath)],
               ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:readme_commit_message', 'result'): [Input(value=Recommended Community Standard: README

Closes: https://github.com/pdxjohnny/testaaaa/issues/108
, definition=repo.readme.git.commit.message)],
               ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:readme_issue', 'result'): [Input(value=https://github.com/pdxjohnny/testaaaa/issues/108, definition=ReadmeIssue)],
               ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:readme_pr_body', 'result'): [Input(value=Closes: https://github.com/pdxjohnny/testaaaa/issues/108, definition=github.pr.body)]},
 'check_for_default_value': [repo.git.base.branch],
 'contexts': [MemoryInputNetworkContextEntry(ctx=Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo), definitions={ReadmeGitRepo: [Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo)], writable.github.remote.origin: [Input(value=origin, definition=writable.github.remote.origin)], repo.git.base.branch: [Input(value=master, definition=repo.git.base.branch)], ReadmePath: [Input(value=/tmp/dffml-feature-git-68ghk7vd/README.md, definition=ReadmePath)], ReadmeIssue: [Input(value=https://github.com/pdxjohnny/testaaaa/issues/108, definition=ReadmeIssue)], repo.readme.git.commit.message: [Input(value=Recommended Community Standard: README

Closes: https://github.com/pdxjohnny/testaaaa/issues/108
, definition=repo.readme.git.commit.message)], github.pr.body: [Input(value=Closes: https://github.com/pdxjohnny/testaaaa/issues/108, definition=github.pr.body)]}, by_origin={('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme', 'result'): [Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo)], ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGitHub:github_owns_remote', 'result'): [Input(value=origin, definition=writable.github.remote.origin)], ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:determin_base_branch', 'result'): [Input(value=master, definition=repo.git.base.branch)], ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:create_readme_file_if_not_exists', 'result'): [Input(value=/tmp/dffml-feature-git-68ghk7vd/README.md, definition=ReadmePath)], ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:readme_issue', 'result'): [Input(value=https://github.com/pdxjohnny/testaaaa/issues/108, definition=ReadmeIssue)], ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:readme_commit_message', 'result'): [Input(value=Recommended Community Standard: README

Closes: https://github.com/pdxjohnny/testaaaa/issues/108
, definition=repo.readme.git.commit.message)], ('alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:readme_pr_body', 'result'): [Input(value=Closes: https://github.com/pdxjohnny/testaaaa/issues/108, definition=github.pr.body)]})],
 'ctx': Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo),
 'dataflow': <dffml.df.types.DataFlow object at 0x7f177ec3b7f0>,
 'definition': repo.git.base.branch,
 'gather': {'base': [],
            'repo': [Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo)]},
 'handle_string': "Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', "
                  "URL='https://github.com/pdxjohnny/testaaaa'), "
                  'definition=ReadmeGitRepo)',
 'input_flow': InputFlow(inputs={'repo': [{'alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme': 'result'}], 'base': ['seed'], 'commit_message': [{'alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:readme_commit_message': 'result'}]}, conditions=[]),
 'input_name': 'base',
 'input_source': 'seed',
 'input_sources': ['seed'],
 'item': Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo),
 'operation': Operation(name='alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:contribute_readme_md', inputs={'repo': ReadmeGitRepo, 'base': repo.git.base.branch, 'commit_message': repo.readme.git.commit.message}, outputs={'result': repo.readme.git.branch}, stage=<Stage.PROCESSING: 'processing'>, conditions=[], expand=[], instance_name='alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:contribute_readme_md', validator=False, retry=0),
 'origin': 'seed',
 'origins': ['seed'],
 'pprint': <function pprint at 0x7f1782d94670>,
 'rctx': <dffml.df.memory.MemoryRedundancyCheckerContext object at 0x7f177ec5e220>,
 'self': <dffml.df.memory.MemoryInputNetworkContext object at 0x7f177ec3ba30>}
> /home/pdxjohnny/Documents/python/dffml/dffml/df/memory.py(788)gather_inputs()
-> return
(Pdb) gather
{'repo': [Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo)], 'base': []}
(Pdb) operation.inputs
{'repo': ReadmeGitRepo, 'base': repo.git.base.branch, 'commit_message': repo.readme.git.commit.message}
(Pdb) self.ctxhd.keys()
dict_keys(["Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo)"])
(Pdb) from pprint import pprint
(Pdb) pprint(inputs.definitions)
{ReadmeGitRepo: [Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo)],
 ReadmeIssue: [Input(value=https://github.com/pdxjohnny/testaaaa/issues/108, definition=ReadmeIssue)],
 ReadmePath: [Input(value=/tmp/dffml-feature-git-68ghk7vd/README.md, definition=ReadmePath)],
 github.pr.body: [Input(value=Closes: https://github.com/pdxjohnny/testaaaa/issues/108, definition=github.pr.body)],
 repo.git.base.branch: [Input(value=master, definition=repo.git.base.branch)],
 repo.readme.git.commit.message: [Input(value=Recommended Community Standard: README

Closes: https://github.com/pdxjohnny/testaaaa/issues/108
, definition=repo.readme.git.commit.message)],
 writable.github.remote.origin: [Input(value=origin, definition=writable.github.remote.origin)]}
(Pdb) gather
{'repo': [Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-68ghk7vd', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo)], 'base': []}
(Pdb) operation.inputs
{'repo': ReadmeGitRepo, 'base': repo.git.base.branch, 'commit_message': repo.readme.git.commit.message}

Suspect discarded because of mismatched origin, if not that, will check definition
- Found out that it was seed vs. output origin mismatch
- Found out that BaseBranch comes from OverlayGit
- Registered OverlayGit as an overlay of OverlayReadme to that it's definitions get loaded
  - This way auto_flow will make the expected origin the output from OverlayGit operations
    rather than seed (the default when no matching outputs are seen on DataFlow init).
- We found it created an infinite loop
  - Will try reusing redundancy checker, that seems to be doing well
alice: please: contribute: recommended community standards: github: GraphQL: was submitted too quickly #1408
Now debugging why readme_pr not called, OverlayGit definitions were seen earlier
on subflow start to be present, must be something else.
- The logs tell us that alice_contribute_readme is returning None, which means
  that the downstream operation is not called, since None means no return value
  in this case.

DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme Outputs: None

Future
- run_custom Optionally support forward subflow
TODO
- Set definition proprety AliceGitRepo.lock to True

44

Found out that util: subprocess: run command events: Do not return after yield of stdout/err
- Fixed in b6eea6e

$ (for i in $(echo determin_base_branch readme_pr_body contribute_readme_md github_owns_remote alice_contribute_readme); do grep -rn "${i} Outputs" .output/2022-07-28-14-11.txt; done) | sort | uniq | sort
354:DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGitHub:github_owns_remote Outputs: {'result': 'origin'}
361:DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:determin_base_branch Outputs: {'result': 'master'}
450:DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:readme_pr_body Outputs: {'result': 'Closes: https://github.com/pdxjohnny/testaaaa/issues/188'}
472:DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:contribute_readme_md Outputs: {'result': 'alice-contribute-recommended-community-standards-readme'}
479:DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme Outputs: None

(Pdb)
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:contribute_readme_md Outputs: {'result': 'alice-contribute-recommended-community-standards-readme'}
(Pdb) pprint(readme_dataflow.flow['alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:readme_pr'].inputs)
{'base': [{'alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:determin_base_branch': 'result'}],
 'body': [{'alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:readme_pr_body': 'result'}],
 'head': [{'alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:contribute_readme_md': 'result'}],
 'origin': ['seed'],
 'repo': [{'alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme': 'result'}],
 'title': [{'alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:readme_pr_title': 'result'}]}

origin is set to seed
- 'origin': ['seed'] was there because OverlayGitHub.github_owns_remote is not in the flow
- We forgot add it to entry_points.txt, added

$ dffml service dev export alice.cli:AlicePleaseContributeCLIDataFlow | tee alice.please.contribute.recommended_community_standards.json
$ (echo -e 'HTTP/1.0 200 OK\n' && dffml dataflow diagram -shortname alice.please.contribute.recommended_community_standards.json) | nc -Nlp 9999;

Opens
- guessed_repo_string_means_no_git_branch_given is feeding git_repo_default_branch but dffml dataflow diagram just have a bug because it's not showing the connection.

graph TD
subgraph a759a07029077edc5c37fea0326fa281[Processing Stage]
style a759a07029077edc5c37fea0326fa281 fill:#afd388b5,stroke:#a4ca7a
subgraph 8cfb8cd5b8620de4a7ebe0dfec00771a[cli_has_repos]
style 8cfb8cd5b8620de4a7ebe0dfec00771a fill:#fff4de,stroke:#cece71
d493c90433d19f11f33c2d72cd144940[cli_has_repos]
e07552ee3b6b7696cb3ddd786222eaad(cmd)
e07552ee3b6b7696cb3ddd786222eaad --> d493c90433d19f11f33c2d72cd144940
cee6b5fdd0b6fbd0539cdcdc7f5a3324(wanted)
cee6b5fdd0b6fbd0539cdcdc7f5a3324 --> d493c90433d19f11f33c2d72cd144940
79e1ea6822bff603a835fb8ee80c7ff3(result)
d493c90433d19f11f33c2d72cd144940 --> 79e1ea6822bff603a835fb8ee80c7ff3
end
subgraph 0c2b64320fb5666a034794bb2195ecf0[cli_is_asking_for_recommended_community_standards]
style 0c2b64320fb5666a034794bb2195ecf0 fill:#fff4de,stroke:#cece71
222ee6c0209f1f1b7a782bc1276868c7[cli_is_asking_for_recommended_community_standards]
330f463830aa97e88917d5a9d1c21500(cmd)
330f463830aa97e88917d5a9d1c21500 --> 222ee6c0209f1f1b7a782bc1276868c7
ba29b52e9c5aa88ea1caeeff29bfd491(result)
222ee6c0209f1f1b7a782bc1276868c7 --> ba29b52e9c5aa88ea1caeeff29bfd491
end
subgraph eac58e8db2b55cb9cc5474aaa402c93e[cli_is_meant_on_this_repo]
style eac58e8db2b55cb9cc5474aaa402c93e fill:#fff4de,stroke:#cece71
6c819ad0228b0e7094b33e0634da9a38[cli_is_meant_on_this_repo]
dc7c5f0836f7d2564c402bf956722672(cmd)
dc7c5f0836f7d2564c402bf956722672 --> 6c819ad0228b0e7094b33e0634da9a38
58d8518cb0d6ef6ad35dc242486f1beb(wanted)
58d8518cb0d6ef6ad35dc242486f1beb --> 6c819ad0228b0e7094b33e0634da9a38
135ee61e3402d6fcbd7a219b0b4ccd73(result)
6c819ad0228b0e7094b33e0634da9a38 --> 135ee61e3402d6fcbd7a219b0b4ccd73
end
subgraph 37887bf260c5c8e9bd18038401008bbc[cli_run_on_repo]
style 37887bf260c5c8e9bd18038401008bbc fill:#fff4de,stroke:#cece71
9d1042f33352800e54d98c9c5a4223df[cli_run_on_repo]
e824ae072860bc545fc7d55aa0bca479(repo)
e824ae072860bc545fc7d55aa0bca479 --> 9d1042f33352800e54d98c9c5a4223df
40109d487bb9f08608d8c5f6e747042f(result)
9d1042f33352800e54d98c9c5a4223df --> 40109d487bb9f08608d8c5f6e747042f
end
subgraph 66ecd0c1f2e08941c443ec9cd89ec589[guess_repo_string_is_directory]
style 66ecd0c1f2e08941c443ec9cd89ec589 fill:#fff4de,stroke:#cece71
737d719a0c348ff65456024ddbc530fe[guess_repo_string_is_directory]
33d806f9b732bfd6b96ae2e9e4243a68(repo_string)
33d806f9b732bfd6b96ae2e9e4243a68 --> 737d719a0c348ff65456024ddbc530fe
dd5aab190ce844673819298c5b8fde76(result)
737d719a0c348ff65456024ddbc530fe --> dd5aab190ce844673819298c5b8fde76
end
subgraph 4ea6696419c4a0862a4f63ea1f60c751[create_branch_if_none_exists]
style 4ea6696419c4a0862a4f63ea1f60c751 fill:#fff4de,stroke:#cece71
502369b37882b300d6620d5b4020f5b2[create_branch_if_none_exists]
fdcb9b6113856222e30e093f7c38065e(name)
fdcb9b6113856222e30e093f7c38065e --> 502369b37882b300d6620d5b4020f5b2
bdcf4b078985f4a390e4ed4beacffa65(repo)
bdcf4b078985f4a390e4ed4beacffa65 --> 502369b37882b300d6620d5b4020f5b2
5a5493ab86ab4053f1d44302e7bdddd6(result)
502369b37882b300d6620d5b4020f5b2 --> 5a5493ab86ab4053f1d44302e7bdddd6
end
subgraph b1d510183f6a4c3fde207a4656c72cb4[determin_base_branch]
style b1d510183f6a4c3fde207a4656c72cb4 fill:#fff4de,stroke:#cece71
476aecd4d4d712cda1879feba46ea109[determin_base_branch]
ff47cf65b58262acec28507f4427de45(default_branch)
ff47cf65b58262acec28507f4427de45 --> 476aecd4d4d712cda1879feba46ea109
150204cd2d5a921deb53c312418379a1(result)
476aecd4d4d712cda1879feba46ea109 --> 150204cd2d5a921deb53c312418379a1
end
subgraph 2a08ff341f159c170b7fe017eaad2f18[git_repo_to_alice_git_repo]
style 2a08ff341f159c170b7fe017eaad2f18 fill:#fff4de,stroke:#cece71
7f74112f6d30c6289caa0a000e87edab[git_repo_to_alice_git_repo]
e58180baf478fe910359358a3fa02234(repo)
e58180baf478fe910359358a3fa02234 --> 7f74112f6d30c6289caa0a000e87edab
9b92d5a346885079a2821c4d27cb5174(result)
7f74112f6d30c6289caa0a000e87edab --> 9b92d5a346885079a2821c4d27cb5174
end
subgraph b5d35aa8a8dcd28d22d47caad02676b0[guess_repo_string_is_url]
style b5d35aa8a8dcd28d22d47caad02676b0 fill:#fff4de,stroke:#cece71
0de074e71a32e30889b8bb400cf8db9f[guess_repo_string_is_url]
c3bfe79b396a98ce2d9bfe772c9c20af(repo_string)
c3bfe79b396a98ce2d9bfe772c9c20af --> 0de074e71a32e30889b8bb400cf8db9f
2a1c620b0d510c3d8ed35deda41851c5(result)
0de074e71a32e30889b8bb400cf8db9f --> 2a1c620b0d510c3d8ed35deda41851c5
end
subgraph 60791520c6d124c0bf15e599132b0caf[guessed_repo_string_is_operations_git_url]
style 60791520c6d124c0bf15e599132b0caf fill:#fff4de,stroke:#cece71
102f173505d7b546236cdeff191369d4[guessed_repo_string_is_operations_git_url]
4934c6211334318c63a5e91530171c9b(repo_url)
4934c6211334318c63a5e91530171c9b --> 102f173505d7b546236cdeff191369d4
8d0adc31da1a0919724baf73d047743c(result)
102f173505d7b546236cdeff191369d4 --> 8d0adc31da1a0919724baf73d047743c
end
subgraph f2c7b93622447999daab403713239ada[guessed_repo_string_means_no_git_branch_given]
style f2c7b93622447999daab403713239ada fill:#fff4de,stroke:#cece71
c8294a87e7aae8f7f9cb7f53e054fed5[guessed_repo_string_means_no_git_branch_given]
5567dd8a6d7ae4fe86252db32e189a4d(repo_url)
5567dd8a6d7ae4fe86252db32e189a4d --> c8294a87e7aae8f7f9cb7f53e054fed5
d888e6b64b5e3496056088f14dab9894(result)
c8294a87e7aae8f7f9cb7f53e054fed5 --> d888e6b64b5e3496056088f14dab9894
end
subgraph 113addf4beee5305fdc79d2363608f9d[github_owns_remote]
style 113addf4beee5305fdc79d2363608f9d fill:#fff4de,stroke:#cece71
049b72b81b976fbb43607bfeeb0464c5[github_owns_remote]
6c2b36393ffff6be0b4ad333df2d9419(remote)
6c2b36393ffff6be0b4ad333df2d9419 --> 049b72b81b976fbb43607bfeeb0464c5
19a9ee483c1743e6ecf0a2dc3b6f8c7a(repo)
19a9ee483c1743e6ecf0a2dc3b6f8c7a --> 049b72b81b976fbb43607bfeeb0464c5
b4cff8d194413f436d94f9d84ece0262(result)
049b72b81b976fbb43607bfeeb0464c5 --> b4cff8d194413f436d94f9d84ece0262
end
subgraph 43a22312a3d4f5c995c54c5196acc50a[create_meta_issue]
style 43a22312a3d4f5c995c54c5196acc50a fill:#fff4de,stroke:#cece71
d2345f23e5ef9f54c591c4a687c24575[create_meta_issue]
1d79010ee1550f057c531130814c40b9(body)
1d79010ee1550f057c531130814c40b9 --> d2345f23e5ef9f54c591c4a687c24575
712d4318e59bd2dc629f0ddebb257ca3(repo)
712d4318e59bd2dc629f0ddebb257ca3 --> d2345f23e5ef9f54c591c4a687c24575
38a94f1c2162803f571489d707d61021(title)
38a94f1c2162803f571489d707d61021 --> d2345f23e5ef9f54c591c4a687c24575
2b22b4998ac3e6a64d82e0147e71ee1b(result)
d2345f23e5ef9f54c591c4a687c24575 --> 2b22b4998ac3e6a64d82e0147e71ee1b
end
subgraph f77af509c413b86b6cd7e107cc623c73[meta_issue_body]
style f77af509c413b86b6cd7e107cc623c73 fill:#fff4de,stroke:#cece71
69a9852570720a3d35cb9dd52a281f71[meta_issue_body]
480d1cc478d23858e92d61225349b674(base)
480d1cc478d23858e92d61225349b674 --> 69a9852570720a3d35cb9dd52a281f71
37035ea5a06a282bdc1e1de24090a36d(readme_issue)
37035ea5a06a282bdc1e1de24090a36d --> 69a9852570720a3d35cb9dd52a281f71
fdf0dbb8ca47ee9022b3daeb8c7df9c0(readme_path)
fdf0dbb8ca47ee9022b3daeb8c7df9c0 --> 69a9852570720a3d35cb9dd52a281f71
428ca84f627c695362652cc7531fc27b(repo)
428ca84f627c695362652cc7531fc27b --> 69a9852570720a3d35cb9dd52a281f71
0cd9eb1ffb3c56d2b0a4359f800b1f20(result)
69a9852570720a3d35cb9dd52a281f71 --> 0cd9eb1ffb3c56d2b0a4359f800b1f20
end
subgraph 8506cba6514466fb6d65f33ace4b0eac[alice_contribute_readme]
style 8506cba6514466fb6d65f33ace4b0eac fill:#fff4de,stroke:#cece71
d4507d3d1c3fbf3e7e373eae24797667[alice_contribute_readme]
68cf7d6869d027ca46a5fb4dbf7001d1(repo)
68cf7d6869d027ca46a5fb4dbf7001d1 --> d4507d3d1c3fbf3e7e373eae24797667
2f9316539862f119f7c525bf9061e974(result)
d4507d3d1c3fbf3e7e373eae24797667 --> 2f9316539862f119f7c525bf9061e974
end
subgraph 4233e6dc67cba131d4ef005af9c02959[contribute_readme_md]
style 4233e6dc67cba131d4ef005af9c02959 fill:#fff4de,stroke:#cece71
3db0ee5d6ab83886bded5afd86f3f88f[contribute_readme_md]
37044e4d8610abe13849bc71a5cb7591(base)
37044e4d8610abe13849bc71a5cb7591 --> 3db0ee5d6ab83886bded5afd86f3f88f
631c051fe6050ae8f8fc3321ed00802d(commit_message)
631c051fe6050ae8f8fc3321ed00802d --> 3db0ee5d6ab83886bded5afd86f3f88f
182194bab776fc9bc406ed573d621b68(repo)
182194bab776fc9bc406ed573d621b68 --> 3db0ee5d6ab83886bded5afd86f3f88f
0ee9f524d2db12be854fe611fa8126dd(result)
3db0ee5d6ab83886bded5afd86f3f88f --> 0ee9f524d2db12be854fe611fa8126dd
end
subgraph a6080d9c45eb5f806a47152a18bf7830[create_readme_file_if_not_exists]
style a6080d9c45eb5f806a47152a18bf7830 fill:#fff4de,stroke:#cece71
67e388f508dd96084c37d236a2c67e67[create_readme_file_if_not_exists]
54faf20bfdca0e63d07efb3e5a984cf1(readme_contents)
54faf20bfdca0e63d07efb3e5a984cf1 --> 67e388f508dd96084c37d236a2c67e67
8c089c362960ccf181742334a3dccaea(repo)
8c089c362960ccf181742334a3dccaea --> 67e388f508dd96084c37d236a2c67e67
5cc65e17d40e6a7223c1504f1c4b0d2a(result)
67e388f508dd96084c37d236a2c67e67 --> 5cc65e17d40e6a7223c1504f1c4b0d2a
end
subgraph e7757158127e9845b2915c16a7fa80c5[readme_commit_message]
style e7757158127e9845b2915c16a7fa80c5 fill:#fff4de,stroke:#cece71
562bdc535c7cebfc66dba920b1a17540[readme_commit_message]
0af5cbea9050874a0a3cba73bb61f892(issue_url)
0af5cbea9050874a0a3cba73bb61f892 --> 562bdc535c7cebfc66dba920b1a17540
2641f3b67327fb7518ee34a3a40b0755(result)
562bdc535c7cebfc66dba920b1a17540 --> 2641f3b67327fb7518ee34a3a40b0755
end
subgraph cf99ff6fad80e9c21266b43fd67b2f7b[readme_issue]
style cf99ff6fad80e9c21266b43fd67b2f7b fill:#fff4de,stroke:#cece71
da44417f891a945085590baafffc2bdb[readme_issue]
d519830ab4e07ec391038e8581889ac3(body)
d519830ab4e07ec391038e8581889ac3 --> da44417f891a945085590baafffc2bdb
268852aa3fa8ab0864a32abae5a333f7(repo)
268852aa3fa8ab0864a32abae5a333f7 --> da44417f891a945085590baafffc2bdb
77a11dd29af309cf43ed321446c4bf01(title)
77a11dd29af309cf43ed321446c4bf01 --> da44417f891a945085590baafffc2bdb
1d2360c9da18fac0b6ec142df8f3fbda(result)
da44417f891a945085590baafffc2bdb --> 1d2360c9da18fac0b6ec142df8f3fbda
end
subgraph 7ec0442cf2d95c367912e8abee09b217[readme_pr]
style 7ec0442cf2d95c367912e8abee09b217 fill:#fff4de,stroke:#cece71
bb314dc452cde5b6af5ea94dd277ba40[readme_pr]
127d77c3047facc1daa621148c5a0a1d(base)
127d77c3047facc1daa621148c5a0a1d --> bb314dc452cde5b6af5ea94dd277ba40
cb421e4de153cbb912f7fbe57e4ad734(body)
cb421e4de153cbb912f7fbe57e4ad734 --> bb314dc452cde5b6af5ea94dd277ba40
cbf7a0b88c0a41953b245303f3e9a0d3(head)
cbf7a0b88c0a41953b245303f3e9a0d3 --> bb314dc452cde5b6af5ea94dd277ba40
e5f9ad44448abd2469b3fd9831f3d159(origin)
e5f9ad44448abd2469b3fd9831f3d159 --> bb314dc452cde5b6af5ea94dd277ba40
a35aee6711d240378eb57a3932537ca1(repo)
a35aee6711d240378eb57a3932537ca1 --> bb314dc452cde5b6af5ea94dd277ba40
dfcce88a7d605d46bf17de1159fbe5ad(title)
dfcce88a7d605d46bf17de1159fbe5ad --> bb314dc452cde5b6af5ea94dd277ba40
a210a7890a7bea8d629368e02da3d806(result)
bb314dc452cde5b6af5ea94dd277ba40 --> a210a7890a7bea8d629368e02da3d806
end
subgraph 227eabb1f1c5cc0bc931714a03049e27[readme_pr_body]
style 227eabb1f1c5cc0bc931714a03049e27 fill:#fff4de,stroke:#cece71
2aea976396cfe68dacd9bc7d4a3f0cba[readme_pr_body]
c5dfd309617c909b852afe0b4ae4a178(readme_issue)
c5dfd309617c909b852afe0b4ae4a178 --> 2aea976396cfe68dacd9bc7d4a3f0cba
40ddb5b508cb5643e7c91f7abdb72b84(result)
2aea976396cfe68dacd9bc7d4a3f0cba --> 40ddb5b508cb5643e7c91f7abdb72b84
end
subgraph 48687c84e69b3db0acca625cbe2e6b49[readme_pr_title]
style 48687c84e69b3db0acca625cbe2e6b49 fill:#fff4de,stroke:#cece71
d8668ff93f41bc241c8c540199cd7453[readme_pr_title]
3b2137dd1c61d0dac7d4e40fd6746cfb(readme_issue)
3b2137dd1c61d0dac7d4e40fd6746cfb --> d8668ff93f41bc241c8c540199cd7453
956e024fde513b3a449eac9ee42d6ab3(result)
d8668ff93f41bc241c8c540199cd7453 --> 956e024fde513b3a449eac9ee42d6ab3
end
subgraph d3ec0ac85209a7256c89d20f758f09f4[check_if_valid_git_repository_URL]
style d3ec0ac85209a7256c89d20f758f09f4 fill:#fff4de,stroke:#cece71
f577c71443f6b04596b3fe0511326c40[check_if_valid_git_repository_URL]
7440e73a8e8f864097f42162b74f2762(URL)
7440e73a8e8f864097f42162b74f2762 --> f577c71443f6b04596b3fe0511326c40
8e39b501b41c5d0e4596318f80a03210(valid)
f577c71443f6b04596b3fe0511326c40 --> 8e39b501b41c5d0e4596318f80a03210
end
subgraph af8da22d1318d911f29b95e687f87c5d[clone_git_repo]
style af8da22d1318d911f29b95e687f87c5d fill:#fff4de,stroke:#cece71
155b8fdb5524f6bfd5adbae4940ad8d5[clone_git_repo]
eed77b9eea541e0c378c67395351099c(URL)
eed77b9eea541e0c378c67395351099c --> 155b8fdb5524f6bfd5adbae4940ad8d5
8b5928cd265dd2c44d67d076f60c8b05(ssh_key)
8b5928cd265dd2c44d67d076f60c8b05 --> 155b8fdb5524f6bfd5adbae4940ad8d5
4e1d5ea96e050e46ebf95ebc0713d54c(repo)
155b8fdb5524f6bfd5adbae4940ad8d5 --> 4e1d5ea96e050e46ebf95ebc0713d54c
6a44de06a4a3518b939b27c790f6cdce{valid_git_repository_URL}
6a44de06a4a3518b939b27c790f6cdce --> 155b8fdb5524f6bfd5adbae4940ad8d5
end
subgraph d3d91578caf34c0ae944b17853783406[git_repo_default_branch]
style d3d91578caf34c0ae944b17853783406 fill:#fff4de,stroke:#cece71
546062a96122df465d2631f31df4e9e3[git_repo_default_branch]
181f1b33df4d795fbad2911ec7087e86(repo)
181f1b33df4d795fbad2911ec7087e86 --> 546062a96122df465d2631f31df4e9e3
57651c1bcd24b794dfc8d1794ab556d5(branch)
546062a96122df465d2631f31df4e9e3 --> 57651c1bcd24b794dfc8d1794ab556d5
5ed1ab77e726d7efdcc41e9e2f8039c6(remote)
546062a96122df465d2631f31df4e9e3 --> 5ed1ab77e726d7efdcc41e9e2f8039c6
4c3cdd5f15b7a846d291aac089e8a622{no_git_branch_given}
4c3cdd5f15b7a846d291aac089e8a622 --> 546062a96122df465d2631f31df4e9e3
end
end
subgraph a4827add25f5c7d5895c5728b74e2beb[Cleanup Stage]
style a4827add25f5c7d5895c5728b74e2beb fill:#afd388b5,stroke:#a4ca7a
end
subgraph 58ca4d24d2767176f196436c2890b926[Output Stage]
style 58ca4d24d2767176f196436c2890b926 fill:#afd388b5,stroke:#a4ca7a
end
subgraph inputs[Inputs]
style inputs fill:#f6dbf9,stroke:#a178ca
128516cfa09b0383023eab52ee24878a(seed<br>dffml.util.cli.CMD)
128516cfa09b0383023eab52ee24878a --> e07552ee3b6b7696cb3ddd786222eaad
ba29b52e9c5aa88ea1caeeff29bfd491 --> cee6b5fdd0b6fbd0539cdcdc7f5a3324
128516cfa09b0383023eab52ee24878a(seed<br>dffml.util.cli.CMD)
128516cfa09b0383023eab52ee24878a --> 330f463830aa97e88917d5a9d1c21500
128516cfa09b0383023eab52ee24878a(seed<br>dffml.util.cli.CMD)
128516cfa09b0383023eab52ee24878a --> dc7c5f0836f7d2564c402bf956722672
ba29b52e9c5aa88ea1caeeff29bfd491 --> 58d8518cb0d6ef6ad35dc242486f1beb
79e1ea6822bff603a835fb8ee80c7ff3 --> e824ae072860bc545fc7d55aa0bca479
135ee61e3402d6fcbd7a219b0b4ccd73 --> e824ae072860bc545fc7d55aa0bca479
40109d487bb9f08608d8c5f6e747042f --> 33d806f9b732bfd6b96ae2e9e4243a68
21ccfd2c550bd853d28581f0b0c9f9fe(seed<br>default.branch.name)
21ccfd2c550bd853d28581f0b0c9f9fe --> fdcb9b6113856222e30e093f7c38065e
dd5aab190ce844673819298c5b8fde76 --> bdcf4b078985f4a390e4ed4beacffa65
9b92d5a346885079a2821c4d27cb5174 --> bdcf4b078985f4a390e4ed4beacffa65
5a5493ab86ab4053f1d44302e7bdddd6 --> ff47cf65b58262acec28507f4427de45
57651c1bcd24b794dfc8d1794ab556d5 --> ff47cf65b58262acec28507f4427de45
4e1d5ea96e050e46ebf95ebc0713d54c --> e58180baf478fe910359358a3fa02234
40109d487bb9f08608d8c5f6e747042f --> c3bfe79b396a98ce2d9bfe772c9c20af
2a1c620b0d510c3d8ed35deda41851c5 --> 4934c6211334318c63a5e91530171c9b
2a1c620b0d510c3d8ed35deda41851c5 --> 5567dd8a6d7ae4fe86252db32e189a4d
5ed1ab77e726d7efdcc41e9e2f8039c6 --> 6c2b36393ffff6be0b4ad333df2d9419
dd5aab190ce844673819298c5b8fde76 --> 19a9ee483c1743e6ecf0a2dc3b6f8c7a
9b92d5a346885079a2821c4d27cb5174 --> 19a9ee483c1743e6ecf0a2dc3b6f8c7a
0cd9eb1ffb3c56d2b0a4359f800b1f20 --> 1d79010ee1550f057c531130814c40b9
dd5aab190ce844673819298c5b8fde76 --> 712d4318e59bd2dc629f0ddebb257ca3
9b92d5a346885079a2821c4d27cb5174 --> 712d4318e59bd2dc629f0ddebb257ca3
e7ad3469d98c3bd160363dbc47e2d741(seed<br>MetaIssueTitle)
e7ad3469d98c3bd160363dbc47e2d741 --> 38a94f1c2162803f571489d707d61021
150204cd2d5a921deb53c312418379a1 --> 480d1cc478d23858e92d61225349b674
1d2360c9da18fac0b6ec142df8f3fbda --> 37035ea5a06a282bdc1e1de24090a36d
5cc65e17d40e6a7223c1504f1c4b0d2a --> fdf0dbb8ca47ee9022b3daeb8c7df9c0
dd5aab190ce844673819298c5b8fde76 --> 428ca84f627c695362652cc7531fc27b
9b92d5a346885079a2821c4d27cb5174 --> 428ca84f627c695362652cc7531fc27b
dd5aab190ce844673819298c5b8fde76 --> 68cf7d6869d027ca46a5fb4dbf7001d1
9b92d5a346885079a2821c4d27cb5174 --> 68cf7d6869d027ca46a5fb4dbf7001d1
150204cd2d5a921deb53c312418379a1 --> 37044e4d8610abe13849bc71a5cb7591
2641f3b67327fb7518ee34a3a40b0755 --> 631c051fe6050ae8f8fc3321ed00802d
2f9316539862f119f7c525bf9061e974 --> 182194bab776fc9bc406ed573d621b68
d2708225c1f4c95d613a2645a17a5bc0(seed<br>repo.directory.readme.contents)
d2708225c1f4c95d613a2645a17a5bc0 --> 54faf20bfdca0e63d07efb3e5a984cf1
2f9316539862f119f7c525bf9061e974 --> 8c089c362960ccf181742334a3dccaea
1d2360c9da18fac0b6ec142df8f3fbda --> 0af5cbea9050874a0a3cba73bb61f892
1daacccd02f8117e67ad3cb8686a732c(seed<br>ReadmeIssueBody)
1daacccd02f8117e67ad3cb8686a732c --> d519830ab4e07ec391038e8581889ac3
2f9316539862f119f7c525bf9061e974 --> 268852aa3fa8ab0864a32abae5a333f7
0c1ab2d4bda10e1083557833ae5c5da4(seed<br>ReadmeIssueTitle)
0c1ab2d4bda10e1083557833ae5c5da4 --> 77a11dd29af309cf43ed321446c4bf01
150204cd2d5a921deb53c312418379a1 --> 127d77c3047facc1daa621148c5a0a1d
40ddb5b508cb5643e7c91f7abdb72b84 --> cb421e4de153cbb912f7fbe57e4ad734
0ee9f524d2db12be854fe611fa8126dd --> cbf7a0b88c0a41953b245303f3e9a0d3
b4cff8d194413f436d94f9d84ece0262 --> e5f9ad44448abd2469b3fd9831f3d159
2f9316539862f119f7c525bf9061e974 --> a35aee6711d240378eb57a3932537ca1
956e024fde513b3a449eac9ee42d6ab3 --> dfcce88a7d605d46bf17de1159fbe5ad
1d2360c9da18fac0b6ec142df8f3fbda --> c5dfd309617c909b852afe0b4ae4a178
1d2360c9da18fac0b6ec142df8f3fbda --> 3b2137dd1c61d0dac7d4e40fd6746cfb
8d0adc31da1a0919724baf73d047743c --> 7440e73a8e8f864097f42162b74f2762
8d0adc31da1a0919724baf73d047743c --> eed77b9eea541e0c378c67395351099c
a6ed501edbf561fda49a0a0a3ca310f0(seed<br>git_repo_ssh_key)
a6ed501edbf561fda49a0a0a3ca310f0 --> 8b5928cd265dd2c44d67d076f60c8b05
8e39b501b41c5d0e4596318f80a03210 --> 6a44de06a4a3518b939b27c790f6cdce
4e1d5ea96e050e46ebf95ebc0713d54c --> 181f1b33df4d795fbad2911ec7087e86
end

As of f8619a6 it works without meta issue
creation: Recommended Community Standard: README pdxjohnny/testaaaa#193

45

$ gif-for-cli --rows $(tput lines) --cols $(tput cols) --export=/mnt/c/Users/Johnny/Downloads/alice-search-alices-adventures-in-wonderland-1.gif "Alice's Adventures in Wonderland"

$ watch -n 0.2 'grep FEEDFACE .output/$(ls .output/ | tail -n 1) | sed -e "s/alice.please.contribute.recommended_community_standards.recommended_community_standards.//g" | grep -i repo'

pdxjohnny · 2022-07-29T12:07:56Z

pdxjohnny
Jul 29, 2022
Maintainer Author

2022-07-29 Engineering Logs

Alice PR: 2nd Party #1401
John's last day before sabbatical
- He will be in town but offline until 2022-08-29
Rolling Alice: 2022 Progress Reports: July Activities Recap: https://youtu.be/JDh2DARl8os
Alice is ready for contribution
- https://github.com/intel/dffml/blob/alice/docs/tutorials/rolling_alice/0001_coach_alice/0002_our_open_source_guide.md
- https://github.com/intel/dffml/blob/alice/entities/alice/CONTRIBUTING.rst
- Self fulfilling prophecy again! We can even automate our contributions to her even if we wanted to! She will eventually! :P
IETF
- Joined SCITT WG, will rejoin in September, others please do as well!
OpenSSF
- Aligned with Identifying Security Threats WG on SCITT looking like a solid direction to cross with Stream 8 for 1-2 year timeframe as Web5 space matures.
Graphics to help people get involved
- https://drive.google.com/drive/folders/1E8tZT15DNjd13jVR6xqsblgLvwTZZo_f

1 reply

pdxjohnny Jul 29, 2022
Maintainer Author

2022-07-29 @pdxjohnny Engineering Logs

AppSec PNW 2022 Talk playlist: https://youtube.com/playlist?list=PLfoJYLR9vr_IAd1vYWdKCOO4YYpGFVv99
- John^2: Living Threat Models are Better Than Dead Threat Models
  - Not yet uploaded but has Alice's first live demo
https://towardsdatascience.com/installing-multiple-alternative-versions-of-python-on-ubuntu-20-04-237be5177474
- $ sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 40
References
- https://tenor.com/search/alice-gifs
- https://tenor.com/view/why-thank-you-thanks-bow-thank-you-alice-in-wonderland-gif-3553903
  - Alice curtsy
- https://tenor.com/view/alice-in-wonderland-gif-26127117
  - Alice blows out unbirthday cake candle

$ alice; sleep 3; gif-for-cli -l 0 --rows $(tput lines) --cols $(tput cols) 3553903

$ gif-for-cli --rows `tput lines` --cols `tput cols` --export=alice-search-alices-adventures-in-wonderland-1.gif "Alice curtsy"
(why-thank-you-thanks-bow-thank-you-alice-in-wonderland-gif-3553903)
$ gif-for-cli --rows `tput lines` --cols `tput cols` --export=ascii-gif-alice-unbirthday-blow-out-candles-0.gif 26127117
$ gif-for-cli --rows `tput lines` --cols `tput cols` ascii-gif-alice-unbirthday-blow-out-candles-0.gif
$ echo gif-for-cli --rows `tput lines` --cols `tput cols`
gif-for-cli --rows 97 --cols 320
$ gif-for-cli -l 0 --rows `tput lines` --cols `tput cols` /mnt/c/Users/Johnny/Downloads/ascii-alices-adventures-in-wonderland-1.gif`

Exploring a Helper Around Run DataFlow run_custom

Realized we already have the lock because it's on git_repository at flow_depth=1

diff --git a/dffml/df/base.py b/dffml/df/base.py
index 4f84c1c7c8..2da0512602 100644
--- a/dffml/df/base.py
+++ b/dffml/df/base.py
@@ -404,14 +404,19 @@ def op(
                     )
 
                 definition_name = ".".join(name_list)
+                print("FEEDFACE", name, definition_name)
                 if hasattr(param_annotation, "__supertype__") and hasattr(
                     param_annotation, "__name__"
                 ):
+                    if "repo" in definition_name:
+                        breakpoint()
                     definition_name = param_annotation.__name__
+                    print("FEEDFACE", name, definition_name)
                 if inspect.isclass(param_annotation) and hasattr(
                     param_annotation, "__qualname__"
                 ):
                     definition_name = param_annotation.__qualname__
+                    print("FEEDFACE", name, definition_name)
 
                 if isinstance(param_annotation, Definition):
                     kwargs["inputs"][name] = param_annotation
diff --git a/dffml/df/types.py b/dffml/df/types.py
index f09a8a3cea..54840f58c0 100644
--- a/dffml/df/types.py
+++ b/dffml/df/types.py
@@ -44,6 +44,7 @@ APPLY_INSTALLED_OVERLAYS = _APPLY_INSTALLED_OVERLAYS()
 
 
 Expand = Union
+LockReadWrite = Union
 
 
 primitive_types = (int, float, str, bool, dict, list, bytes)
@@ -65,7 +66,7 @@ def find_primitive(new_type: Type) -> Type:
     )
 
 
-def new_type_to_defininition(new_type: Type) -> Type:
+def new_type_to_defininition(new_type: Type, lock: bool = False) -> Type:
     """
     >>> from typing import NewType
     >>> from dffml import new_type_to_defininition
@@ -77,6 +78,7 @@ def new_type_to_defininition(new_type: Type) -> Type:
     return Definition(
         name=new_type.__name__,
         primitive=find_primitive(new_type).__qualname__,
+        lock=lock,
         links=(
             create_definition(
                 find_primitive(new_type).__qualname__, new_type.__supertype__
@@ -95,7 +97,28 @@ class CouldNotDeterminePrimitive(Exception):
     """
 
 
-def resolve_if_forward_ref(param_annotation, forward_refs_from_cls):
+DEFAULT_DEFINTION_ANNOTATIONS_HANDLERS = {
+    LockReadWrite: lambda definition: setattr(definition, "lock", True),
+}
+
+
+def resolve_if_forward_ref(
+    param_annotation,
+    forward_refs_from_cls,
+    *,
+    defintion_annotations_handlers=None,
+) -> Tuple[Union["Definition", Any], bool]:
+    """
+    Return values:
+
+        param_or_definition: Union[Definition, Any]
+        lock: bool
+
+            If the definition should be locked or not.
+    """
+    if defintion_annotations_handlers is None:
+        defintion_annotations_handlers = DEFAULT_DEFINTION_ANNOTATIONS_HANDLERS
+    annotations = {}
     if isinstance(param_annotation, ForwardRef):
         param_annotation = param_annotation.__forward_arg__
     if (
@@ -104,11 +127,22 @@ def resolve_if_forward_ref(param_annotation, forward_refs_from_cls):
         and hasattr(forward_refs_from_cls, param_annotation)
     ):
         param_annotation = getattr(forward_refs_from_cls, param_annotation)
+        # Check if are in an annotation
+        param_annotation_origin = get_origin(param_annotation)
+        if param_annotation_origin in defintion_annotations_handlers:
+            annotations[
+                param_annotation_origin
+            ] = defintion_annotations_handlers[param_annotation_origin]
+            param_annotation = list(get_args(param_annotation))[0]
+        # Create definition
         if hasattr(param_annotation, "__name__") and hasattr(
             param_annotation, "__supertype__"
         ):
             # typing.NewType support
-            return new_type_to_defininition(param_annotation)
+            definition = new_type_to_defininition(param_annotation)
+            for handler in annotations.values():
+                handler(definition)
+            return definition
     return param_annotation
 
 
@@ -118,6 +152,7 @@ def _create_definition(
     default=NO_DEFAULT,
     *,
     forward_refs_from_cls: Optional[object] = None,
+    lock: bool = False,
 ):
     param_annotation = resolve_if_forward_ref(
         param_annotation, forward_refs_from_cls
@@ -138,12 +173,14 @@ def _create_definition(
     elif get_origin(param_annotation) in [
         Union,
         collections.abc.AsyncIterator,
+        LockReadWrite,
     ]:
         # If the annotation is of the form Optional
         return create_definition(
             name,
             list(get_args(param_annotation))[0],
             forward_refs_from_cls=forward_refs_from_cls,
+            lock=bool(get_origin(param_annotation) in (LockReadWrite,),),
         )
     elif (
         get_origin(param_annotation) is list
@@ -235,6 +272,7 @@ def create_definition(
     default=NO_DEFAULT,
     *,
     forward_refs_from_cls: Optional[object] = None,
+    lock: bool = False,
 ):
     if hasattr(param_annotation, "__name__") and hasattr(
         param_annotation, "__supertype__"
@@ -246,6 +284,7 @@ def create_definition(
         param_annotation,
         default=default,
         forward_refs_from_cls=forward_refs_from_cls,
+        lock=lock,
     )
     # We can guess name if converting from NewType. However, we can't otherwise.
     if not definition.name:
@@ -847,7 +886,9 @@ class DataFlow:
         for operation in args:
             name = getattr(getattr(operation, "op", operation), "name")
             if name in operations:
-                raise ValueError(f"Operation {name} given as positional and in dict")
+                raise ValueError(
+                    f"Operation {name} given as positional and in dict"
+                )
             operations[name] = operation
 
         self.operations = operations
diff --git a/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py b/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py
index 825f949d65..0ff7e11c31 100644
--- a/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py
+++ b/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py
@@ -8,18 +8,21 @@ import dffml
 import dffml_feature_git.feature.definitions
 
 
-class AliceGitRepo(NamedTuple):
+class AliceGitRepoSpec(NamedTuple):
     directory: str
     URL: str
 
 
+AliceGitRepo = dffml.LockReadWrite[AliceGitRepoSpec]
+
+
 class AliceGitRepoInputSetContextHandle(dffml.BaseContextHandle):
     def as_string(self) -> str:
         return str(self.ctx.repo)
 
 
 class AliceGitRepoInputSetContext(dffml.BaseInputSetContext):
-    def __init__(self, repo: AliceGitRepo):
+    def __init__(self, repo: AliceGitRepoSpec):
         self.repo = repo
 
     async def handle(self) -> AliceGitRepoInputSetContextHandle:

Is this the same as what we had in c89d3d8 ?

    async def alice_contribute_readme(self, repo: AliceGitRepo) -> ReadmeGitRepo:
        key, definition = list(self.parent.op.outputs.items())[0]
        await self.octx.ictx.cadd(
            AliceGitRepoInputSetContext(repo),
            dffml.Input(
                value=repo,
                definition=definition,
                parents=None,
                origin=(self.parent.op.instance_name, key),
            )
        )

diff --git a/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py b/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py
index 825f949d65..1bc1c41e50 100644
--- a/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py
+++ b/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py
@@ -203,30 +203,22 @@ class OverlayREADME:
     ReadmePRBody = NewType("github.pr.body", str)

     # async def cli_run_on_repo(self, repo: "CLIRunOnRepo"):
-    async def alice_contribute_readme(self, repo: AliceGitRepo) -> ReadmeGitRepo:
-        # TODO Clean this up once SystemContext refactor complete
-        readme_dataflow_cls_upstream = OverlayREADME
-        readme_dataflow_cls_overlays = dffml.Overlay.load(
-            entrypoint="dffml.overlays.alice.please.contribute.recommended_community_standards.overlay.readme"
-        )
-        readme_dataflow_upstream = dffml.DataFlow(
-            *dffml.object_to_operations(readme_dataflow_cls_upstream)
-        )
+    async def new_context(self, repo: AliceGitRepo) -> ReadmeGitRepo:
+        return
         # auto_flow with overlays
-        readme_dataflow = dffml.DataFlow(
+        dataflow = dffml.DataFlow(
             *itertools.chain(
                 *[
                     dffml.object_to_operations(cls)
                     for cls in [
-                        readme_dataflow_cls_upstream,
-                        *readme_dataflow_cls_overlays,
+                        upstream,
+                        *overlays,
                     ]
                 ]
             )
         )
         async with dffml.run_dataflow.imp(
-            # dataflow=self.octx.config.dataflow,
-            dataflow=readme_dataflow,
+            dataflow=dataflow,
             input_set_context_cls=AliceGitRepoInputSetContext,
         ) as custom_run_dataflow:
             # Copy all inputs from parent context into child. We eventually
@@ -277,6 +269,18 @@ class OverlayREADME:
                     },
                 )

+    async def alice_contribute_readme(self, repo: AliceGitRepo) -> ReadmeGitRepo:
+        key, definition = list(self.parent.op.outputs.items())[0]
+        await self.octx.ictx.cadd(
+            AliceGitRepoInputSetContext(repo),
+            dffml.Input(
+                value=repo,
+                definition=definition,
+                parents=None,
+                origin=(self.parent.op.instance_name, key),
+            )
+        )
+
     # TODO Run this system context where readme contexts is given on CLI or
     # overriden via disabling of static overlay and application of overlay to
     # generate contents dynamiclly.

Visualize the flow before we attempt to add CONTRIBUTING.md contribution

$ dffml service dev export alice.cli:AlicePleaseContributeCLIDataFlow | tee alice.please.contribute.recommended_community_standards.json
$ (echo -e 'HTTP/1.0 200 OK\n' && dffml dataflow diagram -shortname alice.please.contribute.recommended_community_standards.json) | nc -Nlp 9999;

graph TD
subgraph a759a07029077edc5c37fea0326fa281[Processing Stage]
style a759a07029077edc5c37fea0326fa281 fill:#afd388b5,stroke:#a4ca7a
subgraph 8cfb8cd5b8620de4a7ebe0dfec00771a[cli_has_repos]
style 8cfb8cd5b8620de4a7ebe0dfec00771a fill:#fff4de,stroke:#cece71
d493c90433d19f11f33c2d72cd144940[cli_has_repos]
e07552ee3b6b7696cb3ddd786222eaad(cmd)
e07552ee3b6b7696cb3ddd786222eaad --> d493c90433d19f11f33c2d72cd144940
cee6b5fdd0b6fbd0539cdcdc7f5a3324(wanted)
cee6b5fdd0b6fbd0539cdcdc7f5a3324 --> d493c90433d19f11f33c2d72cd144940
79e1ea6822bff603a835fb8ee80c7ff3(result)
d493c90433d19f11f33c2d72cd144940 --> 79e1ea6822bff603a835fb8ee80c7ff3
end
subgraph 0c2b64320fb5666a034794bb2195ecf0[cli_is_asking_for_recommended_community_standards]
style 0c2b64320fb5666a034794bb2195ecf0 fill:#fff4de,stroke:#cece71
222ee6c0209f1f1b7a782bc1276868c7[cli_is_asking_for_recommended_community_standards]
330f463830aa97e88917d5a9d1c21500(cmd)
330f463830aa97e88917d5a9d1c21500 --> 222ee6c0209f1f1b7a782bc1276868c7
ba29b52e9c5aa88ea1caeeff29bfd491(result)
222ee6c0209f1f1b7a782bc1276868c7 --> ba29b52e9c5aa88ea1caeeff29bfd491
end
subgraph eac58e8db2b55cb9cc5474aaa402c93e[cli_is_meant_on_this_repo]
style eac58e8db2b55cb9cc5474aaa402c93e fill:#fff4de,stroke:#cece71
6c819ad0228b0e7094b33e0634da9a38[cli_is_meant_on_this_repo]
dc7c5f0836f7d2564c402bf956722672(cmd)
dc7c5f0836f7d2564c402bf956722672 --> 6c819ad0228b0e7094b33e0634da9a38
58d8518cb0d6ef6ad35dc242486f1beb(wanted)
58d8518cb0d6ef6ad35dc242486f1beb --> 6c819ad0228b0e7094b33e0634da9a38
135ee61e3402d6fcbd7a219b0b4ccd73(result)
6c819ad0228b0e7094b33e0634da9a38 --> 135ee61e3402d6fcbd7a219b0b4ccd73
end
subgraph 37887bf260c5c8e9bd18038401008bbc[cli_run_on_repo]
style 37887bf260c5c8e9bd18038401008bbc fill:#fff4de,stroke:#cece71
9d1042f33352800e54d98c9c5a4223df[cli_run_on_repo]
e824ae072860bc545fc7d55aa0bca479(repo)
e824ae072860bc545fc7d55aa0bca479 --> 9d1042f33352800e54d98c9c5a4223df
40109d487bb9f08608d8c5f6e747042f(result)
9d1042f33352800e54d98c9c5a4223df --> 40109d487bb9f08608d8c5f6e747042f
end
subgraph 66ecd0c1f2e08941c443ec9cd89ec589[guess_repo_string_is_directory]
style 66ecd0c1f2e08941c443ec9cd89ec589 fill:#fff4de,stroke:#cece71
737d719a0c348ff65456024ddbc530fe[guess_repo_string_is_directory]
33d806f9b732bfd6b96ae2e9e4243a68(repo_string)
33d806f9b732bfd6b96ae2e9e4243a68 --> 737d719a0c348ff65456024ddbc530fe
dd5aab190ce844673819298c5b8fde76(result)
737d719a0c348ff65456024ddbc530fe --> dd5aab190ce844673819298c5b8fde76
end
subgraph 2bcd191634373f4b97ecb9546df23ee5[alice_contribute_contributing]
style 2bcd191634373f4b97ecb9546df23ee5 fill:#fff4de,stroke:#cece71
a2541ce40b2e5453e8e919021011e5e4[alice_contribute_contributing]
3786b4af914402320d260d077844620e(repo)
3786b4af914402320d260d077844620e --> a2541ce40b2e5453e8e919021011e5e4
da4270ecc44b6d9eed9809a560d24a28(result)
a2541ce40b2e5453e8e919021011e5e4 --> da4270ecc44b6d9eed9809a560d24a28
end
subgraph 13b430e6b93de7e40957165687f8e593[contribute_contributing_md]
style 13b430e6b93de7e40957165687f8e593 fill:#fff4de,stroke:#cece71
ff8f8968322872ccc3cf151d167e22a2[contribute_contributing_md]
4f752ce18209f62ed749e88dd1f70266(base)
4f752ce18209f62ed749e88dd1f70266 --> ff8f8968322872ccc3cf151d167e22a2
2def8c6923c832adf33989b26c91295a(commit_message)
2def8c6923c832adf33989b26c91295a --> ff8f8968322872ccc3cf151d167e22a2
f5548fcbcec8745ddf04104fc78e83a3(repo)
f5548fcbcec8745ddf04104fc78e83a3 --> ff8f8968322872ccc3cf151d167e22a2
24292ae12efd27a227a0d6368ba01faa(result)
ff8f8968322872ccc3cf151d167e22a2 --> 24292ae12efd27a227a0d6368ba01faa
end
subgraph 71a5f33f393735fa1cc91419b43db115[contributing_commit_message]
style 71a5f33f393735fa1cc91419b43db115 fill:#fff4de,stroke:#cece71
d034a42488583464e601bcaee619a539[contributing_commit_message]
c0a0fa68a872adf890ed639e07ed5882(issue_url)
c0a0fa68a872adf890ed639e07ed5882 --> d034a42488583464e601bcaee619a539
ce14ca2191f2b1c13c605b240e797255(result)
d034a42488583464e601bcaee619a539 --> ce14ca2191f2b1c13c605b240e797255
end
subgraph db8a1253cc59982323848f5e42c23c9d[contributing_issue]
style db8a1253cc59982323848f5e42c23c9d fill:#fff4de,stroke:#cece71
c39bd2cc88723432048c434fdd337eab[contributing_issue]
821d21e8a69d1fa1757147e7e768f306(body)
821d21e8a69d1fa1757147e7e768f306 --> c39bd2cc88723432048c434fdd337eab
0581b90c76b0a4635a968682b060abff(repo)
0581b90c76b0a4635a968682b060abff --> c39bd2cc88723432048c434fdd337eab
809719538467f6d0bf18f7ae26f08d80(title)
809719538467f6d0bf18f7ae26f08d80 --> c39bd2cc88723432048c434fdd337eab
c9f2ea5a7f25b3ae9fbf5041be5fa071(result)
c39bd2cc88723432048c434fdd337eab --> c9f2ea5a7f25b3ae9fbf5041be5fa071
end
subgraph 1e6046d1a567bf390566b1b995df9dcf[contributing_pr]
style 1e6046d1a567bf390566b1b995df9dcf fill:#fff4de,stroke:#cece71
4ec1433342f2f12ab8c59efab20e7b06[contributing_pr]
bb85c3467b05192c99a3954968c7a612(base)
bb85c3467b05192c99a3954968c7a612 --> 4ec1433342f2f12ab8c59efab20e7b06
77f6c1c6b7ee62881b49c289097dfbde(body)
77f6c1c6b7ee62881b49c289097dfbde --> 4ec1433342f2f12ab8c59efab20e7b06
a0a2fabc65fe5601c7ea289124d04f70(head)
a0a2fabc65fe5601c7ea289124d04f70 --> 4ec1433342f2f12ab8c59efab20e7b06
cf92708915b9f41cb490b991abd6c374(origin)
cf92708915b9f41cb490b991abd6c374 --> 4ec1433342f2f12ab8c59efab20e7b06
210ae36c85f3597c248e0b32da7661ae(repo)
210ae36c85f3597c248e0b32da7661ae --> 4ec1433342f2f12ab8c59efab20e7b06
1700dc637c25bd503077a2a1422142e2(title)
1700dc637c25bd503077a2a1422142e2 --> 4ec1433342f2f12ab8c59efab20e7b06
806e8c455d2bb7ad68112d2a7e16eed6(result)
4ec1433342f2f12ab8c59efab20e7b06 --> 806e8c455d2bb7ad68112d2a7e16eed6
end
subgraph 04c27c13241164ae88456c1377995897[contributing_pr_body]
style 04c27c13241164ae88456c1377995897 fill:#fff4de,stroke:#cece71
a3cebe78451142664930d44ad4d7d181[contributing_pr_body]
6118470d0158ef1a220fe7c7232e1b63(contributing_issue)
6118470d0158ef1a220fe7c7232e1b63 --> a3cebe78451142664930d44ad4d7d181
99a7dd1ae037153eef80e1dee51b9d2b(result)
a3cebe78451142664930d44ad4d7d181 --> 99a7dd1ae037153eef80e1dee51b9d2b
end
subgraph 0d4627f8d8564b6c4ba33c12dcb58fc1[contributing_pr_title]
style 0d4627f8d8564b6c4ba33c12dcb58fc1 fill:#fff4de,stroke:#cece71
bfa172a9399604546048d60db0a36187[contributing_pr_title]
0fd26f9166ccca10c68e9aefa9c15767(contributing_issue)
0fd26f9166ccca10c68e9aefa9c15767 --> bfa172a9399604546048d60db0a36187
77a2f9d4dfad5f520f1502e8ba70e47a(result)
bfa172a9399604546048d60db0a36187 --> 77a2f9d4dfad5f520f1502e8ba70e47a
end
subgraph c67b92ef6a2e025ca086bc2f89d9afbb[create_contributing_file_if_not_exists]
style c67b92ef6a2e025ca086bc2f89d9afbb fill:#fff4de,stroke:#cece71
993a1fe069a02a45ba3579b1902b2a36[create_contributing_file_if_not_exists]
401c179bb30b24c2ca989c64d0b1cdc7(contributing_contents)
401c179bb30b24c2ca989c64d0b1cdc7 --> 993a1fe069a02a45ba3579b1902b2a36
dde78f81b1bdfe02c0a2bf6e51f65cb4(repo)
dde78f81b1bdfe02c0a2bf6e51f65cb4 --> 993a1fe069a02a45ba3579b1902b2a36
e5b8d158dc0ec476dbbd44549a981815(result)
993a1fe069a02a45ba3579b1902b2a36 --> e5b8d158dc0ec476dbbd44549a981815
end
subgraph 4ea6696419c4a0862a4f63ea1f60c751[create_branch_if_none_exists]
style 4ea6696419c4a0862a4f63ea1f60c751 fill:#fff4de,stroke:#cece71
502369b37882b300d6620d5b4020f5b2[create_branch_if_none_exists]
fdcb9b6113856222e30e093f7c38065e(name)
fdcb9b6113856222e30e093f7c38065e --> 502369b37882b300d6620d5b4020f5b2
bdcf4b078985f4a390e4ed4beacffa65(repo)
bdcf4b078985f4a390e4ed4beacffa65 --> 502369b37882b300d6620d5b4020f5b2
5a5493ab86ab4053f1d44302e7bdddd6(result)
502369b37882b300d6620d5b4020f5b2 --> 5a5493ab86ab4053f1d44302e7bdddd6
end
subgraph b1d510183f6a4c3fde207a4656c72cb4[determin_base_branch]
style b1d510183f6a4c3fde207a4656c72cb4 fill:#fff4de,stroke:#cece71
476aecd4d4d712cda1879feba46ea109[determin_base_branch]
ff47cf65b58262acec28507f4427de45(default_branch)
ff47cf65b58262acec28507f4427de45 --> 476aecd4d4d712cda1879feba46ea109
150204cd2d5a921deb53c312418379a1(result)
476aecd4d4d712cda1879feba46ea109 --> 150204cd2d5a921deb53c312418379a1
end
subgraph 2a08ff341f159c170b7fe017eaad2f18[git_repo_to_alice_git_repo]
style 2a08ff341f159c170b7fe017eaad2f18 fill:#fff4de,stroke:#cece71
7f74112f6d30c6289caa0a000e87edab[git_repo_to_alice_git_repo]
e58180baf478fe910359358a3fa02234(repo)
e58180baf478fe910359358a3fa02234 --> 7f74112f6d30c6289caa0a000e87edab
9b92d5a346885079a2821c4d27cb5174(result)
7f74112f6d30c6289caa0a000e87edab --> 9b92d5a346885079a2821c4d27cb5174
end
subgraph b5d35aa8a8dcd28d22d47caad02676b0[guess_repo_string_is_url]
style b5d35aa8a8dcd28d22d47caad02676b0 fill:#fff4de,stroke:#cece71
0de074e71a32e30889b8bb400cf8db9f[guess_repo_string_is_url]
c3bfe79b396a98ce2d9bfe772c9c20af(repo_string)
c3bfe79b396a98ce2d9bfe772c9c20af --> 0de074e71a32e30889b8bb400cf8db9f
2a1c620b0d510c3d8ed35deda41851c5(result)
0de074e71a32e30889b8bb400cf8db9f --> 2a1c620b0d510c3d8ed35deda41851c5
end
subgraph 60791520c6d124c0bf15e599132b0caf[guessed_repo_string_is_operations_git_url]
style 60791520c6d124c0bf15e599132b0caf fill:#fff4de,stroke:#cece71
102f173505d7b546236cdeff191369d4[guessed_repo_string_is_operations_git_url]
4934c6211334318c63a5e91530171c9b(repo_url)
4934c6211334318c63a5e91530171c9b --> 102f173505d7b546236cdeff191369d4
8d0adc31da1a0919724baf73d047743c(result)
102f173505d7b546236cdeff191369d4 --> 8d0adc31da1a0919724baf73d047743c
end
subgraph f2c7b93622447999daab403713239ada[guessed_repo_string_means_no_git_branch_given]
style f2c7b93622447999daab403713239ada fill:#fff4de,stroke:#cece71
c8294a87e7aae8f7f9cb7f53e054fed5[guessed_repo_string_means_no_git_branch_given]
5567dd8a6d7ae4fe86252db32e189a4d(repo_url)
5567dd8a6d7ae4fe86252db32e189a4d --> c8294a87e7aae8f7f9cb7f53e054fed5
d888e6b64b5e3496056088f14dab9894(result)
c8294a87e7aae8f7f9cb7f53e054fed5 --> d888e6b64b5e3496056088f14dab9894
end
subgraph 113addf4beee5305fdc79d2363608f9d[github_owns_remote]
style 113addf4beee5305fdc79d2363608f9d fill:#fff4de,stroke:#cece71
049b72b81b976fbb43607bfeeb0464c5[github_owns_remote]
6c2b36393ffff6be0b4ad333df2d9419(remote)
6c2b36393ffff6be0b4ad333df2d9419 --> 049b72b81b976fbb43607bfeeb0464c5
19a9ee483c1743e6ecf0a2dc3b6f8c7a(repo)
19a9ee483c1743e6ecf0a2dc3b6f8c7a --> 049b72b81b976fbb43607bfeeb0464c5
b4cff8d194413f436d94f9d84ece0262(result)
049b72b81b976fbb43607bfeeb0464c5 --> b4cff8d194413f436d94f9d84ece0262
end
subgraph 8506cba6514466fb6d65f33ace4b0eac[alice_contribute_readme]
style 8506cba6514466fb6d65f33ace4b0eac fill:#fff4de,stroke:#cece71
d4507d3d1c3fbf3e7e373eae24797667[alice_contribute_readme]
68cf7d6869d027ca46a5fb4dbf7001d1(repo)
68cf7d6869d027ca46a5fb4dbf7001d1 --> d4507d3d1c3fbf3e7e373eae24797667
2f9316539862f119f7c525bf9061e974(result)
d4507d3d1c3fbf3e7e373eae24797667 --> 2f9316539862f119f7c525bf9061e974
end
subgraph 4233e6dc67cba131d4ef005af9c02959[contribute_readme_md]
style 4233e6dc67cba131d4ef005af9c02959 fill:#fff4de,stroke:#cece71
3db0ee5d6ab83886bded5afd86f3f88f[contribute_readme_md]
37044e4d8610abe13849bc71a5cb7591(base)
37044e4d8610abe13849bc71a5cb7591 --> 3db0ee5d6ab83886bded5afd86f3f88f
631c051fe6050ae8f8fc3321ed00802d(commit_message)
631c051fe6050ae8f8fc3321ed00802d --> 3db0ee5d6ab83886bded5afd86f3f88f
182194bab776fc9bc406ed573d621b68(repo)
182194bab776fc9bc406ed573d621b68 --> 3db0ee5d6ab83886bded5afd86f3f88f
0ee9f524d2db12be854fe611fa8126dd(result)
3db0ee5d6ab83886bded5afd86f3f88f --> 0ee9f524d2db12be854fe611fa8126dd
end
subgraph a6080d9c45eb5f806a47152a18bf7830[create_readme_file_if_not_exists]
style a6080d9c45eb5f806a47152a18bf7830 fill:#fff4de,stroke:#cece71
67e388f508dd96084c37d236a2c67e67[create_readme_file_if_not_exists]
54faf20bfdca0e63d07efb3e5a984cf1(readme_contents)
54faf20bfdca0e63d07efb3e5a984cf1 --> 67e388f508dd96084c37d236a2c67e67
8c089c362960ccf181742334a3dccaea(repo)
8c089c362960ccf181742334a3dccaea --> 67e388f508dd96084c37d236a2c67e67
5cc65e17d40e6a7223c1504f1c4b0d2a(result)
67e388f508dd96084c37d236a2c67e67 --> 5cc65e17d40e6a7223c1504f1c4b0d2a
end
subgraph e7757158127e9845b2915c16a7fa80c5[readme_commit_message]
style e7757158127e9845b2915c16a7fa80c5 fill:#fff4de,stroke:#cece71
562bdc535c7cebfc66dba920b1a17540[readme_commit_message]
0af5cbea9050874a0a3cba73bb61f892(issue_url)
0af5cbea9050874a0a3cba73bb61f892 --> 562bdc535c7cebfc66dba920b1a17540
2641f3b67327fb7518ee34a3a40b0755(result)
562bdc535c7cebfc66dba920b1a17540 --> 2641f3b67327fb7518ee34a3a40b0755
end
subgraph cf99ff6fad80e9c21266b43fd67b2f7b[readme_issue]
style cf99ff6fad80e9c21266b43fd67b2f7b fill:#fff4de,stroke:#cece71
da44417f891a945085590baafffc2bdb[readme_issue]
d519830ab4e07ec391038e8581889ac3(body)
d519830ab4e07ec391038e8581889ac3 --> da44417f891a945085590baafffc2bdb
268852aa3fa8ab0864a32abae5a333f7(repo)
268852aa3fa8ab0864a32abae5a333f7 --> da44417f891a945085590baafffc2bdb
77a11dd29af309cf43ed321446c4bf01(title)
77a11dd29af309cf43ed321446c4bf01 --> da44417f891a945085590baafffc2bdb
1d2360c9da18fac0b6ec142df8f3fbda(result)
da44417f891a945085590baafffc2bdb --> 1d2360c9da18fac0b6ec142df8f3fbda
end
subgraph 7ec0442cf2d95c367912e8abee09b217[readme_pr]
style 7ec0442cf2d95c367912e8abee09b217 fill:#fff4de,stroke:#cece71
bb314dc452cde5b6af5ea94dd277ba40[readme_pr]
127d77c3047facc1daa621148c5a0a1d(base)
127d77c3047facc1daa621148c5a0a1d --> bb314dc452cde5b6af5ea94dd277ba40
cb421e4de153cbb912f7fbe57e4ad734(body)
cb421e4de153cbb912f7fbe57e4ad734 --> bb314dc452cde5b6af5ea94dd277ba40
cbf7a0b88c0a41953b245303f3e9a0d3(head)
cbf7a0b88c0a41953b245303f3e9a0d3 --> bb314dc452cde5b6af5ea94dd277ba40
e5f9ad44448abd2469b3fd9831f3d159(origin)
e5f9ad44448abd2469b3fd9831f3d159 --> bb314dc452cde5b6af5ea94dd277ba40
a35aee6711d240378eb57a3932537ca1(repo)
a35aee6711d240378eb57a3932537ca1 --> bb314dc452cde5b6af5ea94dd277ba40
dfcce88a7d605d46bf17de1159fbe5ad(title)
dfcce88a7d605d46bf17de1159fbe5ad --> bb314dc452cde5b6af5ea94dd277ba40
a210a7890a7bea8d629368e02da3d806(result)
bb314dc452cde5b6af5ea94dd277ba40 --> a210a7890a7bea8d629368e02da3d806
end
subgraph 227eabb1f1c5cc0bc931714a03049e27[readme_pr_body]
style 227eabb1f1c5cc0bc931714a03049e27 fill:#fff4de,stroke:#cece71
2aea976396cfe68dacd9bc7d4a3f0cba[readme_pr_body]
c5dfd309617c909b852afe0b4ae4a178(readme_issue)
c5dfd309617c909b852afe0b4ae4a178 --> 2aea976396cfe68dacd9bc7d4a3f0cba
40ddb5b508cb5643e7c91f7abdb72b84(result)
2aea976396cfe68dacd9bc7d4a3f0cba --> 40ddb5b508cb5643e7c91f7abdb72b84
end
subgraph 48687c84e69b3db0acca625cbe2e6b49[readme_pr_title]
style 48687c84e69b3db0acca625cbe2e6b49 fill:#fff4de,stroke:#cece71
d8668ff93f41bc241c8c540199cd7453[readme_pr_title]
3b2137dd1c61d0dac7d4e40fd6746cfb(readme_issue)
3b2137dd1c61d0dac7d4e40fd6746cfb --> d8668ff93f41bc241c8c540199cd7453
956e024fde513b3a449eac9ee42d6ab3(result)
d8668ff93f41bc241c8c540199cd7453 --> 956e024fde513b3a449eac9ee42d6ab3
end
subgraph d3ec0ac85209a7256c89d20f758f09f4[check_if_valid_git_repository_URL]
style d3ec0ac85209a7256c89d20f758f09f4 fill:#fff4de,stroke:#cece71
f577c71443f6b04596b3fe0511326c40[check_if_valid_git_repository_URL]
7440e73a8e8f864097f42162b74f2762(URL)
7440e73a8e8f864097f42162b74f2762 --> f577c71443f6b04596b3fe0511326c40
8e39b501b41c5d0e4596318f80a03210(valid)
f577c71443f6b04596b3fe0511326c40 --> 8e39b501b41c5d0e4596318f80a03210
end
subgraph af8da22d1318d911f29b95e687f87c5d[clone_git_repo]
style af8da22d1318d911f29b95e687f87c5d fill:#fff4de,stroke:#cece71
155b8fdb5524f6bfd5adbae4940ad8d5[clone_git_repo]
eed77b9eea541e0c378c67395351099c(URL)
eed77b9eea541e0c378c67395351099c --> 155b8fdb5524f6bfd5adbae4940ad8d5
8b5928cd265dd2c44d67d076f60c8b05(ssh_key)
8b5928cd265dd2c44d67d076f60c8b05 --> 155b8fdb5524f6bfd5adbae4940ad8d5
4e1d5ea96e050e46ebf95ebc0713d54c(repo)
155b8fdb5524f6bfd5adbae4940ad8d5 --> 4e1d5ea96e050e46ebf95ebc0713d54c
6a44de06a4a3518b939b27c790f6cdce{valid_git_repository_URL}
6a44de06a4a3518b939b27c790f6cdce --> 155b8fdb5524f6bfd5adbae4940ad8d5
end
subgraph d3d91578caf34c0ae944b17853783406[git_repo_default_branch]
style d3d91578caf34c0ae944b17853783406 fill:#fff4de,stroke:#cece71
546062a96122df465d2631f31df4e9e3[git_repo_default_branch]
181f1b33df4d795fbad2911ec7087e86(repo)
181f1b33df4d795fbad2911ec7087e86 --> 546062a96122df465d2631f31df4e9e3
57651c1bcd24b794dfc8d1794ab556d5(branch)
546062a96122df465d2631f31df4e9e3 --> 57651c1bcd24b794dfc8d1794ab556d5
5ed1ab77e726d7efdcc41e9e2f8039c6(remote)
546062a96122df465d2631f31df4e9e3 --> 5ed1ab77e726d7efdcc41e9e2f8039c6
4c3cdd5f15b7a846d291aac089e8a622{no_git_branch_given}
4c3cdd5f15b7a846d291aac089e8a622 --> 546062a96122df465d2631f31df4e9e3
end
end
subgraph a4827add25f5c7d5895c5728b74e2beb[Cleanup Stage]
style a4827add25f5c7d5895c5728b74e2beb fill:#afd388b5,stroke:#a4ca7a
end
subgraph 58ca4d24d2767176f196436c2890b926[Output Stage]
style 58ca4d24d2767176f196436c2890b926 fill:#afd388b5,stroke:#a4ca7a
end
subgraph inputs[Inputs]
style inputs fill:#f6dbf9,stroke:#a178ca
128516cfa09b0383023eab52ee24878a(seed<br>dffml.util.cli.CMD)
128516cfa09b0383023eab52ee24878a --> e07552ee3b6b7696cb3ddd786222eaad
ba29b52e9c5aa88ea1caeeff29bfd491 --> cee6b5fdd0b6fbd0539cdcdc7f5a3324
128516cfa09b0383023eab52ee24878a(seed<br>dffml.util.cli.CMD)
128516cfa09b0383023eab52ee24878a --> 330f463830aa97e88917d5a9d1c21500
128516cfa09b0383023eab52ee24878a(seed<br>dffml.util.cli.CMD)
128516cfa09b0383023eab52ee24878a --> dc7c5f0836f7d2564c402bf956722672
ba29b52e9c5aa88ea1caeeff29bfd491 --> 58d8518cb0d6ef6ad35dc242486f1beb
79e1ea6822bff603a835fb8ee80c7ff3 --> e824ae072860bc545fc7d55aa0bca479
135ee61e3402d6fcbd7a219b0b4ccd73 --> e824ae072860bc545fc7d55aa0bca479
40109d487bb9f08608d8c5f6e747042f --> 33d806f9b732bfd6b96ae2e9e4243a68
dd5aab190ce844673819298c5b8fde76 --> 3786b4af914402320d260d077844620e
9b92d5a346885079a2821c4d27cb5174 --> 3786b4af914402320d260d077844620e
150204cd2d5a921deb53c312418379a1 --> 4f752ce18209f62ed749e88dd1f70266
ce14ca2191f2b1c13c605b240e797255 --> 2def8c6923c832adf33989b26c91295a
da4270ecc44b6d9eed9809a560d24a28 --> f5548fcbcec8745ddf04104fc78e83a3
c9f2ea5a7f25b3ae9fbf5041be5fa071 --> c0a0fa68a872adf890ed639e07ed5882
c94383981c3a071b8c3df7293c8c7c92(seed<br>ContributingIssueBody)
c94383981c3a071b8c3df7293c8c7c92 --> 821d21e8a69d1fa1757147e7e768f306
da4270ecc44b6d9eed9809a560d24a28 --> 0581b90c76b0a4635a968682b060abff
90c6a88275f27b28dc12f5741ac1652f(seed<br>ContributingIssueTitle)
90c6a88275f27b28dc12f5741ac1652f --> 809719538467f6d0bf18f7ae26f08d80
150204cd2d5a921deb53c312418379a1 --> bb85c3467b05192c99a3954968c7a612
99a7dd1ae037153eef80e1dee51b9d2b --> 77f6c1c6b7ee62881b49c289097dfbde
24292ae12efd27a227a0d6368ba01faa --> a0a2fabc65fe5601c7ea289124d04f70
b4cff8d194413f436d94f9d84ece0262 --> cf92708915b9f41cb490b991abd6c374
da4270ecc44b6d9eed9809a560d24a28 --> 210ae36c85f3597c248e0b32da7661ae
77a2f9d4dfad5f520f1502e8ba70e47a --> 1700dc637c25bd503077a2a1422142e2
c9f2ea5a7f25b3ae9fbf5041be5fa071 --> 6118470d0158ef1a220fe7c7232e1b63
c9f2ea5a7f25b3ae9fbf5041be5fa071 --> 0fd26f9166ccca10c68e9aefa9c15767
90b3c16d6d8884aa6f70b475d98f661b(seed<br>repo.directory.contributing.contents)
90b3c16d6d8884aa6f70b475d98f661b --> 401c179bb30b24c2ca989c64d0b1cdc7
da4270ecc44b6d9eed9809a560d24a28 --> dde78f81b1bdfe02c0a2bf6e51f65cb4
21ccfd2c550bd853d28581f0b0c9f9fe(seed<br>default.branch.name)
21ccfd2c550bd853d28581f0b0c9f9fe --> fdcb9b6113856222e30e093f7c38065e
dd5aab190ce844673819298c5b8fde76 --> bdcf4b078985f4a390e4ed4beacffa65
9b92d5a346885079a2821c4d27cb5174 --> bdcf4b078985f4a390e4ed4beacffa65
5a5493ab86ab4053f1d44302e7bdddd6 --> ff47cf65b58262acec28507f4427de45
57651c1bcd24b794dfc8d1794ab556d5 --> ff47cf65b58262acec28507f4427de45
4e1d5ea96e050e46ebf95ebc0713d54c --> e58180baf478fe910359358a3fa02234
40109d487bb9f08608d8c5f6e747042f --> c3bfe79b396a98ce2d9bfe772c9c20af
2a1c620b0d510c3d8ed35deda41851c5 --> 4934c6211334318c63a5e91530171c9b
2a1c620b0d510c3d8ed35deda41851c5 --> 5567dd8a6d7ae4fe86252db32e189a4d
5ed1ab77e726d7efdcc41e9e2f8039c6 --> 6c2b36393ffff6be0b4ad333df2d9419
dd5aab190ce844673819298c5b8fde76 --> 19a9ee483c1743e6ecf0a2dc3b6f8c7a
9b92d5a346885079a2821c4d27cb5174 --> 19a9ee483c1743e6ecf0a2dc3b6f8c7a
dd5aab190ce844673819298c5b8fde76 --> 68cf7d6869d027ca46a5fb4dbf7001d1
9b92d5a346885079a2821c4d27cb5174 --> 68cf7d6869d027ca46a5fb4dbf7001d1
150204cd2d5a921deb53c312418379a1 --> 37044e4d8610abe13849bc71a5cb7591
2641f3b67327fb7518ee34a3a40b0755 --> 631c051fe6050ae8f8fc3321ed00802d
2f9316539862f119f7c525bf9061e974 --> 182194bab776fc9bc406ed573d621b68
d2708225c1f4c95d613a2645a17a5bc0(seed<br>repo.directory.readme.contents)
d2708225c1f4c95d613a2645a17a5bc0 --> 54faf20bfdca0e63d07efb3e5a984cf1
2f9316539862f119f7c525bf9061e974 --> 8c089c362960ccf181742334a3dccaea
1d2360c9da18fac0b6ec142df8f3fbda --> 0af5cbea9050874a0a3cba73bb61f892
1daacccd02f8117e67ad3cb8686a732c(seed<br>ReadmeIssueBody)
1daacccd02f8117e67ad3cb8686a732c --> d519830ab4e07ec391038e8581889ac3
2f9316539862f119f7c525bf9061e974 --> 268852aa3fa8ab0864a32abae5a333f7
0c1ab2d4bda10e1083557833ae5c5da4(seed<br>ReadmeIssueTitle)
0c1ab2d4bda10e1083557833ae5c5da4 --> 77a11dd29af309cf43ed321446c4bf01
150204cd2d5a921deb53c312418379a1 --> 127d77c3047facc1daa621148c5a0a1d
40ddb5b508cb5643e7c91f7abdb72b84 --> cb421e4de153cbb912f7fbe57e4ad734
0ee9f524d2db12be854fe611fa8126dd --> cbf7a0b88c0a41953b245303f3e9a0d3
b4cff8d194413f436d94f9d84ece0262 --> e5f9ad44448abd2469b3fd9831f3d159
2f9316539862f119f7c525bf9061e974 --> a35aee6711d240378eb57a3932537ca1
956e024fde513b3a449eac9ee42d6ab3 --> dfcce88a7d605d46bf17de1159fbe5ad
1d2360c9da18fac0b6ec142df8f3fbda --> c5dfd309617c909b852afe0b4ae4a178
1d2360c9da18fac0b6ec142df8f3fbda --> 3b2137dd1c61d0dac7d4e40fd6746cfb
8d0adc31da1a0919724baf73d047743c --> 7440e73a8e8f864097f42162b74f2762
8d0adc31da1a0919724baf73d047743c --> eed77b9eea541e0c378c67395351099c
a6ed501edbf561fda49a0a0a3ca310f0(seed<br>git_repo_ssh_key)
a6ed501edbf561fda49a0a0a3ca310f0 --> 8b5928cd265dd2c44d67d076f60c8b05
8e39b501b41c5d0e4596318f80a03210 --> 6a44de06a4a3518b939b27c790f6cdce
4e1d5ea96e050e46ebf95ebc0713d54c --> 181f1b33df4d795fbad2911ec7087e86
end

Notes
- create_*_if_not_exists doesn't appear connected.
Only either README or CONTRIBUTING is currently being added when
we run with our new CONTRIBUTING contribution flow overlayed.

$ for pr in $(gh -R https://github.com/pdxjohnny/testaaaa pr list --json number --jq '.[].number'); do gh -R https://github.com/pdxjohnny/testaaaa pr close "${pr}"; done
✓ Closed pull request #222 (Recommended Community Standard: README)
✓ Closed pull request #219 (Recommended Community Standard: CONTRIBUTING)
$ nodemon -e py --exec 'clear; for pr in $(gh -R https://github.com/pdxjohnny/testaaaa pr list --json number --jq '.[].number'); do gh -R https://github.com/pdxjohnny/testaaaa pr close "${pr}"; done; (alice please contribute -log debug -repos https://github.com/pdxjohnny/testaaaa -- recommended community standards; gh -R https://github.com/pdxjohnny/testaaaa pr list) 2>&1 | tee .output/$(date +%4Y-%m-%d-%H-%M).txt; test 1'
$ less -S .output/$(ls .output/ | tail -n 1)

Refactor into README and CONTRIBUTING Overlays

Had the thought, aren't we just adding a new context here?

diff --git a/dffml/df/memory.py b/dffml/df/memory.py
index 59286d4927..87c75d637b 100644
--- a/dffml/df/memory.py
+++ b/dffml/df/memory.py
@@ -377,6 +377,7 @@ class MemoryInputNetworkContext(BaseInputNetworkContext):
                     self.ctxhd[handle_string].by_origin[item.origin] = []
                 # Add input to by origin set
                 self.ctxhd[handle_string].by_origin[item.origin].append(item)
+                self.logger.debug("Added to %s: %r", handle_string, item)

     async def uadd(self, *args: Input):
         """
diff --git a/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py b/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py
index 2873a1b193..cc4d374e57 100644
--- a/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py
+++ b/entities/alice/alice/please/contribute/recommended_community_standards/recommended_community_standards.py
@@ -1,7 +1,8 @@
+import asyncio
 import pathlib
 import textwrap
 import itertools
-from typing import NamedTuple, NewType, Optional
+from typing import NamedTuple, NewType, Optional, Type, Any


 import dffml
@@ -183,6 +184,34 @@ class OverlayGitHub:
         return remote


+async def context_adder(
+    self,
+    upstream_cls: Type[Any],
+    input_set_context: dffml.BaseInputSetContext,
+    value: Any,
+):
+    upstream = dffml.DataFlow(*dffml.object_to_operations(upstream_cls))
+    key, definition = list(self.parent.op.outputs.items())[0]
+    async with self.octx.ictx.definitions(self.ctx) as definitions:
+        await self.octx.ictx.cadd(
+            input_set_context,
+            dffml.Input(
+                value=value,
+                definition=definition,
+                parents=None,
+                origin=(self.parent.op.instance_name, key),
+            ),
+            *[
+                item
+                async for item in definitions.inputs()
+                if (
+                    item.definition in upstream.definitions.values()
+                    and item.definition not in self.parent.op.inputs.values()
+                )
+            ],
+        )
+
+
 # NOTE Not sure if the orchestrator will know what to do if we do this
 # ReadmeGitRepo = AliceGitRepo
 class ReadmeGitRepo(NamedTuple):
@@ -204,6 +233,9 @@ class OverlayREADME:

     # async def cli_run_on_repo(self, repo: "CLIRunOnRepo"):
     async def alice_contribute_readme(self, repo: AliceGitRepo) -> ReadmeGitRepo:
+        # await context_adder(
+        #     self, OverlayREADME, AliceGitRepoInputSetContext(repo), repo
+        # )
         # TODO Clean this up once SystemContext refactor complete
         readme_dataflow_cls_upstream = OverlayREADME
         readme_dataflow_cls_overlays = dffml.Overlay.load(

DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:determin_base_branch Stage: PROCESSING: alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:determin_base_branch
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:determin_base_branch Inputs: {'default_branch': 'master'}
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:determin_base_branch Conditions: {}
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:determin_base_branch Outputs: {'result': 'master'}
DEBUG:dffml.MemoryOperationImplementationNetworkContext:---
DEBUG:dffml.MemoryInputNetworkContext:Added to https://github.com/pdxjohnny/testaaaa: Input(value=master, definition=repo.git.base.branch)
DEBUG:dffml.MemoryLockNetworkContext:Acquiring: 6fc55525-c499-421c-8b07-497dd277b1ff(GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa')) (now held by Operation(name='alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayCONTRIBUTING:alice_contribute_contributing', inputs={'repo': AliceGitRepo}, outputs={'result': ContributingGitRepo}, stage=<Stage.PROCESSING: 'processing'>, conditions=[], expand=[], instance_name='alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayCONTRIBUTING:alice_contribute_contributing', validator=False, retry=0))
DEBUG:dffml.MemoryOperationImplementationNetworkContext:---
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayCONTRIBUTING:alice_contribute_contributing Stage: PROCESSING: alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayCONTRIBUTING:alice_contribute_contributing
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayCONTRIBUTING:alice_contribute_contributing Inputs: {'repo': GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa')}
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayCONTRIBUTING:alice_contribute_contributing Conditions: {}
DEBUG:dffml.MemoryInputNetworkContext:Added to GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa'): Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa'), definition=ContributingGitRepo)
DEBUG:dffml.MemoryInputNetworkContext:Added to GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa'): Input(value=origin, definition=writable.github.remote.origin)
DEBUG:dffml.MemoryInputNetworkContext:Added to GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa'): Input(value=master, definition=repo.git.base.branch)
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayCONTRIBUTING:alice_contribute_contributing Outputs: None
DEBUG:dffml.MemoryOperationImplementationNetworkContext:---
DEBUG:dffml.MemoryLockNetworkContext:Acquiring: 6fc55525-c499-421c-8b07-497dd277b1ff(GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa')) (now held by Operation(name='alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:create_branch_if_none_exists', inputs={'repo': AliceGitRepo, 'name': default.branch.name}, outputs={'result': git_branch}, stage=<Stage.PROCESSING: 'processing'>, conditions=[], expand=[], instance_name='alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:create_branch_if_none_exists', validator=False, retry=0))
DEBUG:dffml.MemoryOperationImplementationNetworkContext:---
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:create_branch_if_none_exists Stage: PROCESSING: alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:create_branch_if_none_exists
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:create_branch_if_none_exists Inputs: {'repo': GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa'), 'name': 'main'}
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:create_branch_if_none_exists Conditions: {}
DEBUG:dffml_feature_git.util:proc.create: ('git', 'branch', '-r')
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayGit:create_branch_if_none_exists Outputs: None
DEBUG:dffml.MemoryOperationImplementationNetworkContext:---
DEBUG:dffml.MemoryLockNetworkContext:Acquiring: 6fc55525-c499-421c-8b07-497dd277b1ff(GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa')) (now held by Operation(name='alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme', inputs={'repo': AliceGitRepo}, outputs={'result': ReadmeGitRepo}, stage=<Stage.PROCESSING: 'processing'>, conditions=[], expand=[], instance_name='alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme', validator=False, retry=0))
DEBUG:dffml.MemoryOperationImplementationNetworkContext:---
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme Stage: PROCESSING: alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme Inputs: {'repo': GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa')}
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme Conditions: {}
DEBUG:dffml.MemoryInputNetworkContext:Added to GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa'): Input(value=GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa'), definition=ReadmeGitRepo)
DEBUG:dffml.MemoryInputNetworkContext:Added to GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa'): Input(value=origin, definition=writable.github.remote.origin)
DEBUG:dffml.MemoryInputNetworkContext:Added to GitRepoSpec(directory='/tmp/dffml-feature-git-rrflb9gm', URL='https://github.com/pdxjohnny/testaaaa'): Input(value=master, definition=repo.git.base.branch)
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.recommended_community_standards.OverlayREADME:alice_contribute_readme Outputs: None
DEBUG:dffml.MemoryOperationImplementationNetworkContext:---
DEBUG:dffml.MemoryInputNetworkContext:Received https://github.com/pdxjohnny/testaaaa result {} from <dffml.df.memory.MemoryOrchestratorContext object at 0x7f2405795d90>
DEBUG:dffml.MemoryInputNetworkContext:Received https://github.com/pdxjohnny/testaaaa result {} from <dffml.df.memory.MemoryOrchestratorContext object at 0x7f2405795d90>
DEBUG:dffml.MemoryOperationImplementationNetworkContext:alice.please.contribute.recommended_community_standards.cli.OverlayCLI:cli_run_on_repo Outputs: None
DEBUG:dffml.MemoryOperationImplementationNetworkContext:---
DEBUG:dffml.MemoryOrchestratorContext:ctx.outstanding: 1
DEBUG:dffml.MemoryInputNetworkContext:Received 9eda82af632e2587d31fcd06d5fb0bfb1df47c4a8383e6a998f26c7c4906a86b result {} from <dffml.df.memory.MemoryOrchestratorContext object at 0x7f240588c040>
DEBUG:dffml.MemoryOrchestratorContext:ctx.outstanding: 0
https://github.com/pdxjohnny/testaaaa {}
9eda82af632e2587d31fcd06d5fb0bfb1df47c4a8383e6a998f26c7c4906a86b {}

Want to understand why we are not waiting for the contexts to complete which were added
in above diff and logs.
- Fallback plan is to call both from a function in a separate overlay until it's working
  this will just call run_custom via a helper function for both README and CONTRIBUTING
  overlays.
  - Going to write this first, then contributing new file tutorial
  - Then tutorial on alice shouldi contribute with overlay addition via installed to entrypoint
  - Then test with ability to add overlays via CLI as one offs
  - Final bit of each tutorial is to add to this fallback overlay
- If we still have time before 8 AM then we'll try to debug
alice: please: contribute: recommended community standards: readme: Scope PR title and body definitions
- 1cf1d73
Tried with dffml.run(), it worked right away. Going with this.
- 1bf5e4c
  - alice: please: contribute: recommended community standards: readme: Use dffml.subflow_typecast to execute README contribution
- 85d57ad
  - high level: subflow typecast: Basic OpImpCtx helper
- 8c0531e
  - overlay: static overlay application with loading entrypoint dataflow class with overlays applied.
- af4306a
  - alice: please: contribute: recommended community standards: Disable OverlayMetaIssue in default installed set of overlays

How to help Alice contribute more files

This tutorial will help you create a new Open Architecture / Alice
overlay which runs when another flow runs. The upstream flow
in this case is the AlicePleaseContributeRecommendedCommunityStandards
base flow.

Copy readme overlay to new file

$ cp alice/please/contribute/recommended_community_standards/readme.py alice/please/contribute/recommended_community_standards/contribute.py

Rename types, classes, variables, etc.

$ sed -e 's/Readme/Contributing/g' -e 's/README/CONTRIBUTING/g' -e 's/readme/contributing/g' -i alice/please/contribute/recommended_community_standards/contribute.py

diff --git a/entities/alice/entry_points.txt b/entities/alice/entry_points.txt
index 129b2866a1..9e130cb3b2 100644
--- a/entities/alice/entry_points.txt
+++ b/entities/alice/entry_points.txt
@@ -9,6 +9,7 @@ CLI                                            = alice.please.contribute.recomme
 OverlayGit                                     = alice.please.contribute.recommended_community_standards.recommended_community_standards:OverlayGit
 OverlayGitHub                                  = alice.please.contribute.recommended_community_standards.recommended_community_standards:OverlayGitHub
 OverlayREADME                                  = alice.please.contribute.recommended_community_standards.recommended_community_standards:OverlayREADME
+OverlayCONTRIBUTING                            = alice.please.contribute.recommended_community_standards.recommended_community_standards:OverlayCONTRIBUTING
 # OverlayMetaIssue                               = alice.please.contribute.recommended_community_standards.recommended_community_standards:OverlayMetaIssue

 [dffml.overlays.alice.please.contribute.recommended_community_standards.overlay.readme]

dffml.git/entites/alice/entry_points.txt

[dffml.overlays.alice.please.contribute.recommended_community_standards.overlay.contributing]
OverlayGit                                     = alice.please.contribute.recommended_community_standards.recommended_community_standards:OverlayGit
OverlayGitHub                                  = alice.please.contribute.recommended_community_standards.recommended_community_standards:OverlayGitHu

Reinstall for new entrypoints to take effect

$ python -m pip install -e .

Re-run the command and observe results

for pr in $(gh -R https://github.com/$USER/ pr list --json number --jq '.[].number'); do gh -R https://github.com/pdxjohnny/testaaaa pr close "${pr}"; done; (alice please contribute -log debug -repos https://github.com/pdxjohnny/testaaaa -- recommended community standards; gh -R https://github.com/pdxjohnny/testaaaa pr list

Script to test Coach Alice Our Open Source Guide tutorial

#!/usr/bin/env bash
set -x
set -e

# export USER=githubusername
export REPO_URL="https://github.com/$USER/my-new-python-project"

cd $(mktemp -d)

git clone --depth=1 -b alice https://github.com/intel/dffml dffml
cd dffml/entities/alice
python -m venv .venv
. .venv/bin/activate
python -m pip install -U pip setuptools wheel
python -m pip install \
  -e .[dev] \
  -e ../../ \
  -e ../../examples/shouldi/ \
  -e ../../feature/git/ \
  -e ../../operations/innersource/ \
  -e ../../configloader/yaml/

gh repo create -y --private "${REPO_URL}"
git clone "${REPO_URL}"
cd my-new-python-project
echo 'print("Hello World")' > test.py
git add test.py
git commit -sam 'Initial Commit'
git push --set-upstream origin $(git branch --show-current)
cd ..
rm -rf my-new-python-project

cp alice/please/contribute/recommended_community_standards/readme.py alice/please/contribute/recommended_community_standards/code_of_conduct.py

sed -e 's/Readme/CodeOfConduct/g' -e 's/README/CODE_OF_CONDUCT/g' -e 's/readme/code_of_conduct/g' -i alice/please/contribute/recommended_community_standards/code_of_conduct.py

sed -i 's/OverlayREADME .*/&\nOverlayCODE_OF_CONDUCT                         = alice.please.contribute.recommended_community_standards.code_of_conduct:OverlayCODE_OF_CONDUCT/' entry_points.txt

tee -a entry_points.txt << 'EOF'

[dffml.overlays.alice.please.contribute.recommended_community_standards.code_of_conduct]
OverlayGit                                     = alice.please.contribute.recommended_community_standards.recommended_community_standards:OverlayGit
OverlayGitHub                                  = alice.please.contribute.recommended_community_standards.recommended_community_standards:OverlayGitHub
EOF

python -m pip install -e .

alice please contribute -log debug -repos "${REPO_URL}" -- recommended community standards

gh -R "${REPO_URL}" pr list
# 343     Recommended Community Standard: README  alice-contribute-recommended-community-standards-readme OPEN
# 341     Recommended Community Standard: CONTRIBUTING    alice-contribute-recommended-community-standards-contributing   OPEN
# 339     Recommended Community Standard: CODE_OF_CONDUCT alice-contribute-recommended-community-standards-code_of_conduct        OPEN

for pr in $(gh -R "${REPO_URL}" pr list --json number --jq '.[].number');
do
  gh -R "${REPO_URL}" pr close "${pr}"
done

The Alice codebase

$ find alice/please/ -type f | grep -v __init
alice/please/contribute/recommended_community_standards/contributing.py
alice/please/contribute/recommended_community_standards/cli.py
alice/please/contribute/recommended_community_standards/readme.py
alice/please/contribute/recommended_community_standards/meta_issue.py
alice/please/contribute/recommended_community_standards/recommended_community_standards.py

TODOs

Explain how to add more top level Alice CLI comamnds
Explain how to overlay shouldi flows beyond standard DFFML docs.

pdxjohnny · 2022-08-22T16:45:06Z

pdxjohnny
Aug 22, 2022
Maintainer Author

2022-08-22 Engineering Logs

SCITT

1 reply

pdxjohnny Aug 22, 2022
Maintainer Author

2022-08-22 @pdxjohnny Engineering Logs

SCITT
TODO
- Update with some of the very spotty wording above and try to flush it out with more conceptual meat now that the tone is established / future John has an example to work with.
  - 9aeb7f1
- Update somewhere else in Vol 1 to include from
  - https://github.com/intel/dffml/blob/alice/docs/arch/alice/discussion/0015/reply_0002.md

pdxjohnny · 2022-08-24T23:06:08Z

pdxjohnny
Aug 24, 2022
Maintainer Author

2022-08-24 Engineering Logs

SCITT
- https://mailarchive.ietf.org/arch/msg/scitt/R56CX1LqSgDBRCzZIk3pZnJEV_c/
  - “
    In summary, a NIST Vulnerability Disclosure Report (VDR) is an attestation
    by a software vendor showing that the vendor has checked each component of a
    software product SBOM for vulnerabilities and reports on the details of any
    vulnerabilities reported by a NIST NVD search. The VDR is a living document
    which the software vendor updates as needed when new vulnerabilities have
    been discovered and reported. A VDR is published whenever a software vendor
    issues a new or updated SBOM, including initial product release, making it
    available online, all the time, to all customers of the product described in
    the VDR. This gives software consumers that ability to answer the question
    "What is the vulnerability status of my software product from Vendor V, as
    of NOW?".”
  - From VEX to VDR? Lets dive in more next week

2 replies

sedihglow Aug 25, 2022

2022-08-24 @sedihglow Engineering Logs

Alice
- Ran through contributing setup on local PC
  - https://github.com/intel/dffml/blob/alice/entities/alice/CONTRIBUTING.rst#cloning-the-repo
alice please build if needed and run /path/to/repo
- Try two different repos, mainly focused on C
  - https://github.com/sedihglow/rpi4
  - https://github.com/sedihglow/red_black_tree

$ sudo update-alternatives: using /usr/bin/python3.9 to provide /usr/local/bin/python (python) in auto mode
$ sudo apt-get update && sudo apt-get install -y tmux python3.9 python3-pip python3.9-venv python3.9-dev build-essential

pdxjohnny Aug 29, 2022
Maintainer Author

https://datatracker.ietf.org/doc/draft-ietf-rats-architecture/

pdxjohnny · 2022-08-29T15:01:13Z

pdxjohnny
Aug 29, 2022
Maintainer Author

Policy
- ABC’s of Conformity Assessment
  - https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.2000-01.pdf
  - This might be helpful later when we write docs for / think about how to apply policy (see vol 0 introduction arch diagram)
SCITT
- Zachary Newman shared looking at OpenSSF / SCITT terminology ran into same topics that we did when we brought up using shared underlying protocols and formats in the 2022-07-25 SCITT meeting when talking about RATs style attestation vs SLSA/in-toto/sigstore style.
  - https://mailarchive.ietf.org/arch/msg/scitt/utSOqlCifoorbqUGWNf-wMlBYR4/
    - Dick agrees with Zach's analysis. "I've also been monitoring the OpenSSF Scorecard initiative, which goes beyond sigstore attestation checking to assign a "trust score". Not sure if this has traction, but there is a lot of activity on github. https://github.com/ossf/scorecard/blob/main/README.md#basic-usage OpenSSF does NOT appear to be following/implementing NIST C-SCRM recommendations and standards for Executive Order 14028 and consumer software labeling and other attestation recommendations; https://www.nist.gov/document/software-supply-chain-security-guidance-under-executive-order-eo-14028-section-4e" [Dick Brooks]
- Commit message to charter
  - The Endor POC by @OR13 was exemplary because there was a low amount of abstraction / extra information / steps introduced for the learner to understand the sequence of data transformations involved. It makes clear the contents of the serialization of choice (DIDs + VCs in Endor's case) and how that varies across the steps. The POC provided immediate value on the mailing list in a way that examples which introduce more abstraction layers are unable to do as quickly.
    
    We apply our recent learning from this success by adding to the charter the production of a similar example which in this patch we call "file-based", but we could change that to a more descriptive term if there is one. Having an example similar to the learning methodology presented via Endor would accelerate the pace at which developers up and down the software stack and in different programming languages would be able to adopt SCITT. This is due to the low level of abstraction introduced by it's file and shell based implementation. Files and shell commands translate easy into other languages where they can be slowly swapped out from initial fork/exec and equivalents to language code.
    
    The SCITT community could potentially provide documentation on how the fork/exec style implementation could be transformed into the HTTP server implementation. Due to the generic nature of SCITT and the many touchpoints various software systems will likely have with it in the future. It is important for us to consider as a part of our threat model the effect cohesive example documentation has on the correctness of downstream implementations. Providing cohesive examples where we start with the basics (file-based), moving to an example environment implementers are likely to be working in (HTTP-based), and finally explaining how we went from the basic to the complex would give a robust view of what SCITT should look like to implementers and provide them with a clear path to a hopefully correct implementation.
    
    More cohesive documentation will reduce the number of security vulnerabilities we see in our communities code. Code which is fundamentally about security in nature. This modification to the charter seeks to act on recent learnings around example code experienced within the SCITT community itself and seeks to contribute to the development of our threat model as we think about SCITT's lifecycle and rollout.
- For this reason I propose we
- where they will be creating the future of SCITT's robust, actively maintained solutions.
  - https://mailarchive.ietf.org/arch/msg/scitt/Hz9BSiIN7JHAgsZL6MuDHK4p7P8/
  - https://github.com/OR13/endor
    - This is learning methodology goldmine.
- Increase level of focus and depth to final milestone about documentation / example implementations ietf-scitt/charter#21
- https://mailarchive.ietf.org/arch/msg/scitt/B9cwkueu3gdQ7lBKkhILcFLD0E4/
RATS
- https://datatracker.ietf.org/doc/draft-ietf-rats-architecture/
SBOM
- We [DFFML community] intend to use the "living" SBOM VDR capabilities to facilitate the breathing of life into our living threat models. This will allow us to facilitate vulns on architecture.
  - https://spdx.github.io/spdx-spec/v2.3/
  - https://energycentral.com/c/pip/what-nist-sbom-vulnerability-disclosure-report-vdr
  - The recommendation by NIST to provide software consumers with a NIST VDR is gaining traction as a best practice. The latest version of the SPDX SBOM standard, version 2.3, includes provisions (K.1.9) enabling a software vendor to associate a specific SBOM document for a software product with its online NIST VDR attestation for that product, which is linked within the SBOM. The link refers to a “living” SBOM VDR document that is updated by a software vendor, whenever new vulnerabilities are reported. Having this “always updated NIST VDR” available enables software consumers to answer the question “What is the vulnerability status of my software product from Vendor V, as of NOW?”, providing consumers with on-going, up-to-date visibility into the risks that may be present in an installed software product, as new vulnerabilities (CVE's) are being reported/released.
    
    As stated previously, NIST did not prescribe a format for a NIST VDR attestation, but guidance is provided on what a VDR includes. Reliable Energy Analytics (REA) has produced an open-source “interpretation” of what a NIST VDR contains in order to meet EO 14028, which is available here in an XML Schema format with samples provided in XML and JSON (https://raw.githubusercontent.com/rjb4standards/REA-Products/master/SBOMVDR_JSON/VDR_118.json) formats.

1 reply

pdxjohnny · 2022-08-30T16:46:19Z

pdxjohnny
Aug 30, 2022
Maintainer Author

2022-08-30 Engineering Logs

3 replies

pdxjohnny Aug 31, 2022
Maintainer Author

https://github.com/opensbom-generator/spdx-sbom-generator

pdxjohnny Aug 31, 2022
Maintainer Author

https://huggingface.co/spaces/huggingface/diffuse-the-rest

2024-05-24T06:02:07Z

github-actions[bot]
bot May 24, 2024

2024-05-24 Engineering Logs

0 replies

2024-05-25T06:02:15Z

github-actions[bot]
bot May 25, 2024

2024-05-25 Engineering Logs

0 replies

2024-05-26T06:02:17Z

github-actions[bot]
bot May 26, 2024

2024-05-26 Engineering Logs

0 replies

2024-05-27T06:03:09Z

github-actions[bot]
bot May 27, 2024

2024-05-27 Engineering Logs

0 replies

2024-05-28T06:02:15Z

github-actions[bot]
bot May 28, 2024

2024-05-28 Engineering Logs

0 replies

pdxjohnny · 2024-05-29T06:02:19Z

github-actions[bot]
bot May 29, 2024

2024-05-29 Engineering Logs

1 reply

pdxjohnny May 30, 2024
Maintainer Author

TODO
- Least privilege in LTM representation

2024-05-30T06:02:07Z

github-actions[bot]
bot May 30, 2024

2024-05-30 Engineering Logs

0 replies

pdxjohnny · 2024-05-31T06:02:42Z

github-actions[bot]
bot May 31, 2024

2024-05-31 Engineering Logs

1 reply

pdxjohnny Jun 1, 2024
Maintainer Author

https://github.com/pdm-project/pdm
https://docs.redpanda.com/redpanda-connect/guides/getting_started/
- https://docs.redpanda.com/redpanda-connect/components/inputs/http_server/
- Agent pipelines combine with policy engine

docker run --rm docker.redpanda.com/redpandadata/redpanda connect create > ./connect.yaml

pdxjohnny · 2024-06-01T06:01:58Z

github-actions[bot]
bot Jun 1, 2024

2024-06-01 Engineering Logs

1 reply

pdxjohnny Jun 1, 2024
Maintainer Author

Federated transparent statements enable the policy engine to distribute execution of jobs and their stacks.
- WIP: orchestrator: distribuited: NatsOrchestrator #772
- df: Orchestrator: Distribuited orchestrator using Nats as message queue #790

2024-06-02T06:02:18Z

github-actions[bot]
bot Jun 2, 2024

2024-06-02 Engineering Logs

0 replies

pdxjohnny · 2024-06-03T06:02:19Z

github-actions[bot]
bot Jun 3, 2024

2024-06-03 Engineering Logs

1 reply

pdxjohnny Jun 4, 2024
Maintainer Author

2024-06-03 @pdxjohnny Engineering Logs

Manifest based comms with LLM

Metaschema translation across ontologies

Transparency Service secured ontologies

Use Case: Attestations of alignment to S2C2F and org overlays ietf-scitt/use-cases#18

dffml/docs/discussions/alice_engineering_comms/0243/reply_0000.md

Lines 19 to 117 in aa6442f

    
           """ 
        
           License: BSD 3-Clause 
        
           This example demonstrates how to use a custom global URL opener installed with `urllib.request.install_opener` to block access to URLs. 
        
           - References 
        
             - https://rdflib.readthedocs.io/en/stable/apidocs/rdflib.plugins.parsers.html#module-rdflib.plugins.parsers.jsonld 
        
           from rdflib import Graph, URIRef, Literal 
        
           test_json = ''' 
        
           { 
        
               "@context": { 
        
                   "dc": "http://purl.org/dc/terms/", 
        
                   "rdf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#", 
        
                   "rdfs": "http://www.w3.org/2000/01/rdf-schema#" 
        
               }, 
        
               "@id": "http://example.org/about", 
        
               "dc:title": { 
        
                   "@language": "en", 
        
                   "@value": "Someone's Homepage" 
        
               } 
        
           } 
        
           ''' 
        
           g = Graph().parse(data=test_json, format='json-ld') 
        
           list(g) == [(URIRef('http://example.org/about'), 
        
               URIRef('http://purl.org/dc/terms/title'), 
        
               Literal("Someone's Homepage", lang='en'))] 
        
             - https://rdflib.readthedocs.io/en/stable/_modules/examples/secure_with_urlopen.html 
        
               - Upstream 
        
           """ 
        
           import http.client 
        
           import logging 
        
           import os 
        
           import sys 
        
           from typing import Optional 
        
           from urllib.request import HTTPHandler, OpenerDirector, Request, install_opener 
        
           from rdflib import Graph 
        
           class SecuredHTTPHandler(HTTPHandler): 
        
               def http_open(self, req: Request) -> http.client.HTTPResponse: 
        
                   # TODO Protect with SCITT 
        
                   # - https://scitt.io/distributing-with-oci-scitt.html 
        
                   # TODO Query the registry 
        
                   # - https://oras.land/ 
        
                   #   - https://github.com/oras-project/oras-credentials-go 
        
                   # - https://github.com/oras-project/oras-py 
        
                   #   - https://oras-project.github.io/oras-py/ 
        
                   #     - https://docs.python.org/3/library/unittest.mock.html#patch-object 
        
                   if req.get_full_url().endswith("blocked.jsonld"): 
        
                       raise PermissionError("Permission denied for URL") 
        
                   return super().http_open(req) 
        
           def main() -> None: 
        
               logging.basicConfig( 
        
                   level=os.environ.get("PYTHON_LOGGING_LEVEL", logging.INFO), 
        
                   stream=sys.stderr, 
        
                   datefmt="%Y-%m-%dT%H:%M:%S", 
        
                   format=( 
        
                       "%(asctime)s.%(msecs)03d %(process)d %(thread)d %(levelno)03d:%(levelname)-8s " 
        
                       "%(name)-12s %(module)s:%(lineno)s:%(funcName)s %(message)s" 
        
                   ), 
        
               ) 
        
               opener = OpenerDirector() 
        
               opener.add_handler(SecuredHTTPHandler()) 
        
               install_opener(opener) 
        
               graph = Graph() 
        
               # Attempt to parse a JSON-LD document that will result in the blocked URL 
        
               # being accessed. 
        
               error: Optional[PermissionError] = None 
        
               try: 
        
                   graph.parse( 
        
                       data=r"""{ 
        
                       "@context": "http://example.org/blocked.jsonld", 
        
                       "@id": "example:subject", 
        
                       "example:predicate": { "@id": "example:object" } 
        
                   }""", 
        
                       format="json-ld", 
        
                   ) 
        
               except PermissionError as caught: 
        
                   logging.info("Permission denied: %s", caught) 
        
                   error = caught 
        
               # `Graph.parse` would have resulted in a `PermissionError` being raised from 
        
               # the url opener. 
        
               assert isinstance(error, PermissionError) 
        
               assert error.args[0] == "Permission denied for URL" 
        
           if __name__ == "__main__": 
        
               main()

pdxjohnny · 2024-06-04T06:02:08Z

github-actions[bot]
bot Jun 4, 2024

2024-06-04 Engineering Logs

1 reply

pdxjohnny Jun 4, 2024
Maintainer Author

2024-06-04 @pdxjohnny Engineering Logs

https://github.com/bottlerocket-os/bottlerocket
- OS for containers
https://brandur.org/two-phase-render
- N+1 query problem
A viewpoint on software supply chain security, are we getting lost in translation?
- https://www.researchgate.net/publication/375626773_A_Viewpoint_on_Software_Supply_Chain_Security_Are_We_Getting_Lost_in_Translation
- https://www.researchgate.net/
  - Created account to download
  - https://www.sciencedirect.com/science/article/pii/S0264275123003827
  - https://apps.dtic.mil/sti/tr/pdf/ADA383563.pdf
    - we showed that student eye movements could serve to predict when they were going to make errors and to disambiguate the interpretation of various actions

pdxjohnny · 2024-06-05T06:02:25Z

github-actions[bot]
bot Jun 5, 2024

2024-06-05 Engineering Logs

1 reply

pdxjohnny Jun 5, 2024
Maintainer Author

https://magazine.sebastianraschka.com/p/understanding-large-language-models

pdxjohnny · 2024-06-06T06:02:33Z

github-actions[bot]
bot Jun 6, 2024

2024-06-06 Engineering Logs

1 reply

pdxjohnny Jun 6, 2024
Maintainer Author

2024-06-06 @pdxjohnny Engineering Logs

https://github.com/directus/directus
- df: overlay: Implement middleware/RBAC chains of ordered applications of overlays #1400
GitHub custom properties are world readable
https://henryjacksonsociety.org/publications/lessons-from-the-first-cyberwar-how-supporting-ukraine-on-the-digital-battlefield-can-help-improve-the-uks-online-resilience/
https://docs.directus.io/self-hosted/config-options.html#file-storage
- STORAGE_LOCATIONS
  - A CSV of storage locations (e.g., local,digitalocean,amazon) to use. You can use any names you'd like for these keys.
    - https://github.com/s3fs-fuse/s3fs-fuse
      - https://docs.digitalocean.com/reference/api/spaces-api/
        
        DO doesn't do event notifications
https://docs.directus.io/self-hosted/sso-examples.html#keycloak
tpm2-tss warnings on wget pdxjohnny/pdxjohnny#1
https://forgejo.org/download/
database path is not relative to APP_DATA_PATH go-gitea/gitea#3816
bug: working dir path its not fully honored with GITEA_WORK_DIR go-gitea/gitea#5482 (comment)

GITEA_WORK_DIR=$HOME/.local/appdata forgejo web

http://localhost:3000/.well-known/openid-configuration

{
    "issuer": "http://localhost:3000/",
    "authorization_endpoint": "http://localhost:3000/login/oauth/authorize",
    "token_endpoint": "http://localhost:3000/login/oauth/access_token",
    "jwks_uri": "http://localhost:3000/login/oauth/keys",
    "userinfo_endpoint": "http://localhost:3000/login/oauth/userinfo",
    "introspection_endpoint": "http://localhost:3000/login/oauth/introspect",
    "response_types_supported": [
        "code",
        "id_token"
    ],
    "id_token_signing_alg_values_supported": [
        "RS256"
    ],
    "subject_types_supported": [
        "public"
    ],
    "scopes_supported": [
        "openid",
        "profile",
        "email",
        "groups"
    ],
    "claims_supported": [
        "aud",
        "exp",
        "iat",
        "iss",
        "sub",
        "name",
        "preferred_username",
        "profile",
        "picture",
        "website",
        "locale",
        "updated_at",
        "email",
        "email_verified",
        "groups"
    ],
    "code_challenge_methods_supported": [
        "plain",
        "S256"
    ],
    "grant_types_supported": [
        "authorization_code",
        "refresh_token"
    ]
}

https://docs.directus.io/self-hosted/sso-examples.html#github
http://localhost:3000/admin/applications/oauth2
https://github.com/directus/directus/blob/b5c4c6dc7c65af75a7e1e8d9cba5e1cdc2b1901e/docs/self-hosted/sso.md?plain=1#L61
- 1. On Authorized redirect URIs put your Directus instance address plus /auth/login/google/callback. For example, you should put
    https://directus.myserver.com/auth/login/google/callback where
    https://directus.myserver.com should be the address of your Directus instance.
    If you are testing locally you should add
    http://localhost:8055/auth/login/google/callback too

https://docs.directus.io/getting-started/quickstart.html

export HOST_IP="$(ip address show docker0 | grep inet | awk '{print $2}' | sed -e 's/\/.*//g')" && \
echo 'using --net host so localhost' && \
export HOST_IP="localhost" && \
docker run \
  -ti \
  --rm \
  -p 8055:8055 \
  -e AUTH_FORGEJO_REDIRECT_ALLOW_LIST="http://localhost:8055/auth/login/forgejo" \
  -e AUTH_FORGEJO_DRIVER="oauth2" \
  -e AUTH_FORGEJO_AUTHORIZE_URL="http://${HOST_IP}:3000/login/oauth/authorize" \
  -e AUTH_FORGEJO_ACCESS_URL="https://${HOST_IP}:3000/login/oauth/access_token" \
  -e AUTH_FORGEJO_PROFILE_URL="https://${HOST_IP}:3000/api/user" \
  -e AUTH_FORGEJO_CLIENT_ID="..." \
  -e AUTH_FORGEJO_CLIENT_SECRET="..." \
  -e AUTH_PROVIDERS="forgejo" \
  -e AUTH_DISABLE_DEFAULT=true \
  -e PUBLIC_URL=https://localhost:3000 \
  -e AUTH_FOREGOJO_ALLOW_PUBLIC_REGISTRATION=true \
  --net host \
  -e SECRET="$(head -n 99999 /dev/urandom | sha384sum - | awk '{print $1}')" \
  directus/directus

TODO
- Why is it insecure to bind 0.0.0.0?
  - We should bind to UNIX sockets when possible or the loopback if not (127.0.0.1). Anyone on your local network scan connect to 0.0.0.0.

2024-06-07T06:02:28Z

github-actions[bot]
bot Jun 7, 2024

2024-06-07 Engineering Logs

0 replies

Alice Engineering Comms 🪬 #1406

pdxjohnny Jul 18, 2022 Maintainer

Engineering Log Process

Replies: 649 comments · 476 replies

pdxjohnny Jul 18, 2022 Maintainer Author

2022-07-18 Engineering Logs

pdxjohnny Jul 18, 2022 Maintainer Author

2022-07-18 @pdxjohnny Engineering Logs

pdxjohnny Jul 19, 2022 Maintainer Author

2022-07-19 Engineering Logs

pdxjohnny Mar 7, 2023 Maintainer Author

2022-07-19 @pdxjohnny Engineering Logs

pdxjohnny Jul 20, 2022 Maintainer Author

2022-07-20 Engineering Logs

pdxjohnny Jul 20, 2022 Maintainer Author

2022-07-20 @pdxjohnny Engineering Logs

pdxjohnny Jul 20, 2022 Maintainer Author

2022-07-20 Identifying Security Threats WG

pdxjohnny Jul 21, 2022 Maintainer Author

2022-07-21 Engineering Logs

pdxjohnny Jul 23, 2022 Maintainer Author

2022-07-23

pdxjohnny Jul 24, 2022 Maintainer Author

2022-07-28 Alice Intelligence/Open Architecture Working Group Initial Meeting

pdxjohnny Jul 25, 2022 Maintainer Author

2022-07-25 Engineering Logs

pdxjohnny Jul 25, 2022 Maintainer Author

2022-07-25 @pdxjohnny Engineering Logs

pdxjohnny Jul 25, 2022 Maintainer Author

2022-07-25 Supply Chain Integrity, Transparency and Trust (SCITT)

pdxjohnny Jul 26, 2022 Maintainer Author

2022-07-26 Engineering Logs

pdxjohnny Jul 26, 2022 Maintainer Author

2022-07-26 @pdxjohnny Engineering Logs

pdxjohnny Jul 27, 2022 Maintainer Author

2022-07-27 Engineering Logs

pdxjohnny Jul 28, 2022 Maintainer Author

2022-07-27 @pdxjohnny Engineering Logs

Refactoring and Thinking About Locking of Repos for Contributions

pdxjohnny Jul 28, 2022 Maintainer Author

2022-07-28 Engineering Logs

pdxjohnny Jul 28, 2022 Maintainer Author

2022-07-28 @pdxjohnny Engineering Logs

43

44

45

pdxjohnny Jul 29, 2022 Maintainer Author

2022-07-29 Engineering Logs

pdxjohnny Jul 29, 2022 Maintainer Author

2022-07-29 @pdxjohnny Engineering Logs

Exploring a Helper Around Run DataFlow run_custom

Refactor into README and CONTRIBUTING Overlays

How to help Alice contribute more files

Script to test Coach Alice Our Open Source Guide tutorial

TODOs

pdxjohnny Aug 22, 2022 Maintainer Author

2022-08-22 Engineering Logs

pdxjohnny Aug 22, 2022 Maintainer Author

2022-08-22 @pdxjohnny Engineering Logs

pdxjohnny Aug 24, 2022 Maintainer Author

2022-08-24 Engineering Logs

sedihglow Aug 25, 2022

2022-08-24 @sedihglow Engineering Logs

pdxjohnny Aug 29, 2022 Maintainer Author

pdxjohnny Aug 29, 2022 Maintainer Author

pdxjohnny Aug 29, 2022 Maintainer Author

2022-08-29 @pdxjohnny Engineering Logs

pdxjohnny
Jul 18, 2022
Maintainer

Replies: 649 comments 476 replies

pdxjohnny
Jul 18, 2022
Maintainer Author

pdxjohnny Jul 18, 2022
Maintainer Author

pdxjohnny
Jul 19, 2022
Maintainer Author

pdxjohnny Mar 7, 2023
Maintainer Author

pdxjohnny
Jul 20, 2022
Maintainer Author

pdxjohnny Jul 20, 2022
Maintainer Author

pdxjohnny Jul 20, 2022
Maintainer Author

pdxjohnny
Jul 21, 2022
Maintainer Author

pdxjohnny
Jul 23, 2022
Maintainer Author

pdxjohnny
Jul 24, 2022
Maintainer Author

pdxjohnny
Jul 25, 2022
Maintainer Author

pdxjohnny Jul 25, 2022
Maintainer Author

pdxjohnny Jul 25, 2022
Maintainer Author

pdxjohnny
Jul 26, 2022
Maintainer Author

pdxjohnny Jul 26, 2022
Maintainer Author

pdxjohnny
Jul 27, 2022
Maintainer Author

pdxjohnny Jul 28, 2022
Maintainer Author

pdxjohnny
Jul 28, 2022
Maintainer Author

pdxjohnny Jul 28, 2022
Maintainer Author

pdxjohnny
Jul 29, 2022
Maintainer Author

pdxjohnny Jul 29, 2022
Maintainer Author

pdxjohnny
Aug 22, 2022
Maintainer Author

pdxjohnny Aug 22, 2022
Maintainer Author

pdxjohnny
Aug 24, 2022
Maintainer Author

pdxjohnny Aug 29, 2022
Maintainer Author

pdxjohnny
Aug 29, 2022
Maintainer Author

pdxjohnny Aug 29, 2022
Maintainer Author