Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

schemas registry and refernce to schemas #63

Open
lubagov opened this issue Oct 17, 2021 · 5 comments
Open

schemas registry and refernce to schemas #63

lubagov opened this issue Oct 17, 2021 · 5 comments

Comments

@lubagov
Copy link

lubagov commented Oct 17, 2021

I'm looking at the library for now, trying to figure out the possibility of using schema references. Is this supported in the library?
For example I have 2 files:

com.myapp.Children.avsc:

{
  "namespace": "com.myapp",
  "type": "record",
  "name": "Children",
  "fields": [
    {"name": "first_name", "type": "string"},
    {"name": "last_name", "type": "string"}
  ]
}

com.myapp.People.avsc:

{
  "namespace": "com.myapp",
  "type": "record",
  "name": "People",
  "fields": [
    {"name": "id", "type": "string"},
    {"name": "first_name", "type": "string"},
    {"name": "last_name", "type": "string"},
    {
      "name": "children",
      "type": ["null",
        {
          "type": "array",
          "items": "Children"
        }],
      "doc": ""
    }
  ]
}

And I want to get a schema, that matches People.avsc.

My test code:

        $schemaRegistryClient = new CachedRegistry(
            new PromisingRegistry(
                new Client(['base_uri' => 'http://****:8081'])
            ),
            new AvroObjectCacheAdapter()
        );

        $recordSerializer = new RecordSerializer(
            $schemaRegistryClient,
            [
                // If you want to auto-register missing schemas set this to true
                RecordSerializer::OPTION_REGISTER_MISSING_SCHEMAS => false,
                // If you want to auto-register missing subjects set this to true
                RecordSerializer::OPTION_REGISTER_MISSING_SUBJECTS => false,
            ]
        );

        $baseDir = "./avro";

        $inflector = static function ($record, bool $isKey) {
            echo "Searching for:".get_class($record);
            $ext = $isKey ? '.key.avsc' : '.avsc';
            $fileName = is_object($record)
                ? str_replace('\\', '.', get_class($record))
                : 'default';

            return $fileName . $ext;
        };

        $resolver = new FileResolver($baseDir, $inflector);
        $normalizer = new GetSetMethodNormalizer();
        $encoder = new AvroSerDeEncoder($recordSerializer);

        $symfonySerializer = new Serializer([$normalizer], [$encoder]);

        $children=new Children("Nikolai","Lubiagov");

        $people=new People("Nikolai","Lubiagov", [$children]);

        $schemac= $resolver->valueSchemaFor($children);
        $schemap= $resolver->valueSchemaFor($people);

I don't see any code at all to resolve the link. Or how to transfer the already obtained Children scheme to People, is this supported?

I also can't get already registread schema, with refernces over:
$sc=$schemaRegistryClient->schemaForSubjectAndVersion("phptopic-value",1)->wait();

{
"subject":"phptopic-value",
"version":1,
"id":70,
"references":[
{
"name":"Children",
"subject":"Children",
"version":1
}
],
"schema":"{"type":"record","name":"People","namespace":"com.myapp","fields":[{"name":"id","type":"string"},{"name":"first_name","type":"string"},{"name":"last_name","type":"string"},{"name":"children","type":["null",{"type":"array","items":"Children"}],"doc":""}]}"
}

I found this 322fcd6 for previously defined types, but if type defined in another subject, it is not fit...?

@lubagov lubagov changed the title Registry of schemas, and refernce to schemas. schemas registry and refernce to schemas Oct 17, 2021
@BeyerJC
Copy link

BeyerJC commented Oct 29, 2021

Is that what you mean ? There are some good examples in the test Filder.

https://github.com/flix-tech/avro-serde-php/blob/master/test/Objects/Schema/Generation/Fixture/RecordWithRecordType.php

@lubagov
Copy link
Author

lubagov commented Oct 29, 2021

I mean schema references. When one schema is link to another schema. This example forms one schema with 2 types. I also need Protobuf and it's not there.

(I was under the impression that it's easier to make your own library than trying to adapt that)

The annotated method also seems to be of little use when using several clients implemented in different programming languages? Seems, no sense to describe it, it's easier to describe the once in AVRO / Protobuf, and generate the code...

@tPl0ch
Copy link
Collaborator

tPl0ch commented Oct 30, 2021

@lubagov you are correct in that the tying this library does not support schema references from other topics right now. In general I am planning on adding these features, not only references, but also Protobuf & JSON schemas. It was on the plate for over a year now, but I believe that I can finally find time for the implementation.

I will leave this open as a reference.

@AlexOstrovsky
Copy link

AlexOstrovsky commented Sep 11, 2023

@tPl0ch any update on possibily to reference schemas from other topics?

@tPl0ch
Copy link
Collaborator

tPl0ch commented Jan 26, 2024

@AlexOstrovsky currently there is no time for me to work on this, I am happily accepting PRs that will add features, but this project currently has no priority. Sorry. :(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants