Google Summer of Code 2024 application

[shaheemazmalmmd]

Hello all!

FOSSology as an org is planning to apply for Google Summer of Code this year as well. In preparation to do so, we are creating this discussion to gather ideas for the GSoC-2024 projects.

Since last year, there are some changes made by Google in the program. You can read more about it on their website. But as a summary,

• Three sized projects are acceptable by program:
  • Small sized projects (~90 hour)
  • Medium sized projects (~175 hours)
  • Large projects (~350 hours)
• The program is open to 18 years and older people.
  • No longer limited to college students only.
• Flexible time lines between 12 and 22 weeks.

Please feel free to drop any questions here (or start a new discussion) you have regarding the program, you want to submit a proposal idea, you want to be mentor in the program, etc.

While submitting a project idea, please tag it to be "Medium" or "Large" sized and who you'll prefer working on it "student" or "professional".
Some resources from Google:

• https://google.github.io/gsocguides/mentor/defining-a-project-ideas-list
• https://google.github.io/gsocguides/mentor/making-your-ideas-page

You can also check the detailed documentation of previous students from GSoC-2023.

[shaheemazmalmmd]

Some useful links for contributors who Are first time here :

• Basic workflow - https://www.youtube.com/watch?v=TZqU5ZALI7U
• Installation - https://github.com/fossology/fossology/wiki/Install-from-Source
• Good first issues - https://github.com/fossology/fossology/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22
• Slack channel - https://fossology.slack.com/

[shaheemazmalmmd]

Idea!

Title: Improving FOSSology CI scanner image

Goal: Enhancing current scanner image with new reports and features

As a fun project, FOSSology started combining scanners in a simple and small Docker image which can be run on CI providers. The image is currently capable of understanding build environment (GitLab/GitHub Actions/Travis) and use their API's to fetch diff of a branch or scan the complete repo. The capabilities of image include license scanning with Nomos and ojo scanners, copyright and keyword scanning with respective scanners. The image makes use of a Python script to perform all the tasks.

1. The integration with GitHub Actions can be improved by reporting line number where a license violation is found.
2. Allowing user to provide a different list of Keywords for scanning (currently stored at /usr/local/share/fossology/keyword/agent/keyword.conf).
3. Improving on whitelist format with feature to provide it from other sources, currently it is read from a file which is expected to be in the root of repo being scanned.
4. Ability to download a dependency for scan (path provided at pipeline trigger).

Additionally, the JSON output of nomos needs to be enhanced providing highlight and line information in the output.

Category	Rating
Low Hanging Fruit	-
Risk/Exploratory	**
Fun/Periphial	***
Core Development	**
Project Infrastructure	***
Project size	Large
Preferred contributor	Student/Professional
Skills needed	Docker, Python

Contact: @GMishx @shaheemazmalmmd

[shubha987]

I want to work on this idea . What should i have to do for it?

[sjha2048]

@shaheemazmalmmd for JSON output can this help?
https://github.com/ynqa/jnv

[shaheemazmalmmd]

Idea!

Title: Support text phrases and bulk based scanning for MONK a like agent

Goal: Adding text phrases from UI to database and use existing bulk phrases and provide ability to scan them using MONK and identify files if the match is 100%.

FLOW :

• Create a UI Where user can add multiple text phrases associated with license(FROM FOSSology License Database).
• Use existing bulk phrases table from database.
• Create a new agent like existing MONK agent which not only identifies the matches but also decides the files.
• Test cases needs to be provided as well.

Category	Rating
Low Hanging Fruit	**
Risk/Exploratory	*
Fun/Periphial	**
Core Development	*
Project Infrastructure	**
Project size	Medium
Preferred contributor	Student/professional
Skills needed	PHP, C++

Contact: @GMishx @shaheemazmalmmd

[abhinavs001]

I want to work on this idea. I am unable to join the Slack channel, @GMishx please guide me through

[GMishx]

Idea!

Title: SPDX license expression support

Goal: Support SPDX license expression detection and reporting

1. Support SPDX license expressions in FOSSology such as MIT AND (GPL-2.0-only OR BSD-2-Clause).
2. Differentiate SPDX licenses with exceptions. FOSSology currently stores license exceptions as licenses.
3. Scanning SPDX Expressions with ojo as step 1.
4. UI components to create and see license expressions in file clearing page.
5. Updating reports to export the expressions correctly.

Category	Rating
Low Hanging Fruit	-
Risk/Exploratory	**
Fun/Periphial	***
Core Development	***
Project Infrastructure	***
Project size	Large
Preferred contributor	-
Skills needed	PHP, C/C++

Contact: @GMishx @shaheemazmalmmd

[GMishx]

Idea!

Title: Overhauling scheduler design

Goal: Improving FOSSology scheduler or replacing with OTS solution

The existing scheduler design is causing new issues which need to be addressed. Moreover, existing scheduler design is not touched in years.

Concerning points

1. The scheduler is written in C which makes it next to impossible to find cause of a failure.
2. The C language does not support exception handling out of the box. It makes code less readable and prone to errors.
3. The linear queue design causes issue when there should be only one instance of an agent running for an upload, but overall the agent is not mutually exclusive.

   For example, if the monkbulk has a limit set to 1, it should be implied for only single upload. But with linear queue, this monkbulk job will block all other agents from executing even when they are not effected by the results of monkbulk.
   This essentially makes the agent mutually exclusive even though, there is a special flag EXCLUSIVE for the very same purpose: https://github.com/fossology/fossology/wiki/Job-Scheduler#agentconfs

• One idea on redesigning the queue, it can be broken into buckets per upload each maintaining its own priority queue. There can be another queue for global operations like maintenance, delagent, etc.
• Doing so, each bucket can be traversed in round-robin and pick first pending job and check against host limit. This will eliminate the scenario mentioned in point 3. Also, exclusive agents can be sent to global queue.

    upload specific queue
  |-<upload_2> -> nomos, copyright, ojo, keyword
  |-<upload_3> -> monkbulk, decider, monkbulk, decider
  |-<upload_4> -> reuser, decider
  
  global queue
  -> delagent,
  

4. Since the FOSSology is released, there can be number of new scheduling libraries being released which needs to be explored. They can be a nice addition to the project.

Category	Rating
Low Hanging Fruit	-
Risk/Exploratory	**
Fun/Periphial	***
Core Development	***
Project Infrastructure	*
Project size	Large
Preferred contributor	Professional
Skills needed	C/C++, Go, any fast language

**Contact: @GMishx @ag4ums @avinal **

[avinal]

Good one. I do think we need to document a blueprint of what the new drop-in is supposed to do. Breaking this project into chunks would be a good idea too.

[ag4ums]

This is technically the most fun project; it's like heart surgery, you cannot go wrong :)

[GMishx]

Idea!

Title: Debian packaging for Debian repository

Goal: Improve Debian packaging and make it acceptable for APT

The existing effort to put FOSSology under Debian packaging list needs to be taken forward. A repository under Debian Salsa was setup initially but not maintained any more: https://salsa.debian.org/fossology-team/fossology
It is configured to use gbp.

Blockers

1. The Debian building mechanism does not allow installation from sources other than apt. The Composer packages need to be packed as Debian packages and shipped with FOSSology.
2. Packaging and shipping other tools needs to satisfy their licensing terms.
3. The versions of packages in APT and actual versions used are different.
4. APT also provides JS libraries like JQuery and DataTables but RHL does not.

See also

• feat(buildsystem): add CMake Buildsytem #2075
• https://wiki.debian.org/PackagingWithGit
• https://wiki.debian.org/SimplePackagingTutorial
• https://wiki.debian.org/Diagrams
• https://wiki.debian.org/PHP
• https://peertube.debian.social/videos/watch/0fb2dbc4-f43d-477e-8b14-20c426f970de

Category	Rating
Low Hanging Fruit	*
Risk/Exploratory	**
Fun/Periphial	***
Core Development	*
Project Infrastructure	***
Project size	Small
Preferred contributor	Student/Professional
Skills needed	Debian, APT, CMake

Contact: @GMishx @shaheemazmalmmd

[sjha2048]

@GMishx what about an unofficial package?

[GMishx]

What do you mean by unofficial package here @sjha2048 ?

We already publish packages over release pages. But idea here is to get fossology in debian apt again.

[GMishx]

Idea!

Title: REST API improvements

Goal: Completing REST API implementation and migrate to v2

• Writing test cases for all the existing and new functionalities.
• Improve REST API and expose more endpoints
  • Update REST API to v2
  • feat(api): migrate /tokens endpoints to v2 #2617
  • feat(api): (1) Upgrade Upload APIs to Version 2  #2633
  • feat(api): (2) Upgrade upload APIs to Version 2 #2634
  • https://github.com/orgs/fossology/projects/2/views/2

Category	Rating
Low Hanging Fruit	**
Risk/Exploratory	*
Fun/Periphial	**
Core Development	***
Project Infrastructure	**
Project size	Large
Preferred contributor	Student/Professional
Skills needed	PHP, Slim framework

Contact: @GMishx @shaheemazmalmmd

[Ibrahim-Muhammad13]

can you please point out to existing related issues?

[valens200]

Interested.❤️

[NicolasToussaint]

Idea!

Title: New Artificial Intelligence based copyright and license scanner agent

Goal: Integrate new AI capabilities to improve the scanners

A quick ChatGPT 3.5 test shows it's pretty good at spotting the correct licenses and expressing them as SPDX expressions.

One idea would be integrate a similar technology into an Fossology agent - however not by relying on external proprietary services but by building a dedicated LLM model based on existing open source solutions.

Category	Rating
Low Hanging Fruit	-
Risk/Exploratory	***
Fun/Peripheral	***
Core Development	*
Project Infrastructure	*
Project size	Large
Preferred contributor	Student/Professional

Contact: @NicolasToussaint @ag4ums

[ag4ums]

LLM based (LLaMA-2) scanners/finder for both copyright and license identification/correction .

[hastagAB]

Very Interesting idea! We can fine tune few SOTA LLMs over a custom layer for our use case.
I would be delighted to be a mentor for this idea.

[ShreyaGautamm]

Hello mentors, I am excited about the proposed project and interested to work and contribute to the same. I have previously worked on LLMs and have collaborated with teams to develop projects & publish papers in esteemed conferences like ACL and EMNLP. Please find the links to the papers:
ACL’23: https://aclanthology.org/2023.acl-long.581/
EMNLP’23: https://aclanthology.org/2023.emnlp-main.136/
I’m also currently working on a LLM project around UML development of software.
I’ll be reading more about the technicalities of the proposed project and will come up with few queries if I have any.
(This text is NOT written by any LLM!)

[sjha2048]

would love to collaborate with @hastagAB on this one
Maybe we can use something like https://github.com/ollama/ollama to bundle a smaller model locally for installation.

[GMishx]

Idea!

Title: Support SPDX 3.0 reports

Goal: Support reading and generating SPDX 3.0 reports

• Support generation of SPDX 3.0 reports in different formats.
• Start with Core, Software and Licensing profiles. Increase if needed.
• Support generation of SPDX reports in JSON format.
• Support ingestion of SPDX 3.0 reports

Refs:

• https://github.com/spdx/spdx-3-model/
• https://github.com/fossology/fossology/tree/master/src/spdx2
• https://github.com/fossology/fossology/tree/master/src/reportImport

Category	Rating
Low Hanging Fruit	**
Risk/Exploratory	**
Fun/Periphial	*
Core Development	**
Project Infrastructure	*
Project size	Large
Preferred contributor	Student/Professional
Skills needed	PHP, Twig

Contact: @GMishx @shaheemazmalmmd

[shaheemazmalmmd]

Idea!

Title: Support text phrases and bulk phrases scanning for MONK agent

Goal: Adding text phrases from UI to database and provide ability to scan them using MONK.

FLOW : Create a UI Where user can add multiple text phrases associated with license from UI. and user should have a option to choose bulk license text based on users or whole. Also there should be another Tab where a user can select bulk phrases with two options one with whole users or selected users and another with whole bulk table.

Should enable MONK or New MONK like agent to scan from phrases tables. also enabling options to add license text acknowledgement and comments from UI.

Test cases needs to be provided as well.

Category	Rating
Low Hanging Fruit	**
Risk/Exploratory	*
Fun/Peripheral	**
Core Development	*
Project Infrastructure	**
Project size	Large
Preferred contributor	Student/professional
Skills needed	C++, PHP, Twig or Python

[shaheemazmalmmd]

Idea!

Title: Data pipelining for safaa project.

Goal: Automate the process of model training using pipelining.

FLOW : Currently in Safaa Project data was manually curated And we see that most of the things are manual here. the projuct should concentrate on creating a pipeline, Utilizing LLMS if required to increase the accuracy, use deep learning techniques to improve.

Test cases needs to be provided as well.

Category	Rating
Low Hanging Fruit	**
Risk/Exploratory	*
Fun/Peripheral	**
Core Development	*
Project Infrastructure	**
Project size	Large
Preferred contributor	Student/professional
Skills needed	Python, ML And Data

Contact : @ag4ums