Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New project type: Question and answer #304

Open
wants to merge 42 commits into
base: master
Choose a base branch
from
Open

New project type: Question and answer #304

wants to merge 42 commits into from

Conversation

guillim
Copy link
Contributor

@guillim guillim commented Jul 23, 2019

Summary:
This pull request has a simple objective: creating a new annotation mode, for the task "question and answer"

Ref: #81

What changed

  • Backend: 2 models were created (QandAProject & QandAAnnotation ) and all the following related (serializer)
    The model Document was slightly modified, adding the text field called extra_text - allows some tasks to store additional information, like a question for instance.
    Migrations were created following the model changes as well
    Questionable: the necessity of having QandAStorage. At the moment, I guess I could use Seq2seqStorage instead.

  • Frontend: a new front was added using the component download_qanda.vue. Visual examples for uploading / downloading data were created as well.

Other Info
This is a Draft pull request at the moment. No tests were created yet, and it isn't merged with the latest master branch due to a problem with pyodbc on my side

Guillim and others added 23 commits June 18, 2019 17:21
…, it doesn't work yet. we simply put the barebone.
…e ability to have questions associated to a text. This way, we can simply think annotation as a two step task, entirely separated:

- create a question => sequence to sequence [already there, but has some 
issues to be resolved]
- answer a question => question and answer [which i am coding at the 
moment]
… is covered as well.

It needs as well to bring extra_text as an optional field.
"components//question_and_answer.vue
27:6  Invalid attribute separator found"
@guillim guillim marked this pull request as ready for review July 30, 2019 12:13
@guillim
Copy link
Contributor Author

guillim commented Jul 30, 2019

@Hironsan If you have time to review this, it would be very nice. From the issue #81 it sounded like many people were waiting for this feature. Let me know if you see improvement I could do!

…a rea email, we will have to do set up the parameter EMAIL_HOST etc..
Copy link
Member

@c-w c-w left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a preliminary pass-through of the code without running the branch just to get a conversation started about the pull request. I'm a bit concerned that this introduces quite a lot of new concepts when really the Q&A task is a combination of sequence labeling and document classification. I'm wondering if it would be possible to re-use some of the existing frontend functionality and remix the sequence labeling Vue code with the document classification Vue code. Perhaps via a mixin or something like that?

app/server/static/package-lock.json Outdated Show resolved Hide resolved
app/server/static/components/question_and_answer.vue Outdated Show resolved Hide resolved
app/server/static/components/question_and_answer.vue Outdated Show resolved Hide resolved
app/server/static/components/question_and_answer.vue Outdated Show resolved Hide resolved
app/server/static/components/annotator_qanda.vue Outdated Show resolved Hide resolved
app/server/static/components/annotator_qanda.vue Outdated Show resolved Hide resolved
app/api/migrations/0004_question_and_answer_creation.py Outdated Show resolved Hide resolved
app/api/managers.py Show resolved Hide resolved
app/api/models.py Show resolved Hide resolved
@guillim
Copy link
Contributor Author

guillim commented Aug 8, 2019

Don't know exactly why... but the new database mssql fails on Travis CI. It successfully passes sqlite and postgres databases though... Any clue on how to fix this @c-w ?

@icoxfog417
Copy link
Contributor

django-pyodbc-azure==2.1.0.0 seems not be installed. This requirement is added recently so you have to apply this commit to your branch.

@guillim
Copy link
Contributor Author

guillim commented Aug 9, 2019

django-pyodbc-azure==2.1.0.0 seems not be installed. This requirement is added recently so you have to apply this commit to your branch.

still having the issue, any other clue ?

@c-w
Copy link
Member

c-w commented Aug 12, 2019

@guillim The current build failure no longer is related to the environment but instead is a model error:

django.db.utils.ProgrammingError: ('42000', "[42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Column 'text' in table 'api_qandaannotation' is of a type that is invalid for use as a key column in an index. (1919) (SQLExecDirectW)")

I'd try to take a look at the schema created by manage.py migrate and see what limitations SQL server has that must be worked around.

…e index 'api_qandaannotation_document_id_user_id_response_start_offset_cfb14664_uniq' exceeds the maximum length of 1700 bytes for nonclustered indexes"

So we change from a response of maximum 1000 character to a response of 
500 character. Note: this could be problematic in case someone wants su 
submit a whole text as an aswer in case the text is larger than 500 
charcaters for instance
@guillim
Copy link
Contributor Author

guillim commented Aug 13, 2019

Ok. I finally found out what was the issue with Microsoft SQL Server: they don't accept TextField when used as a unique_together property

So I changed it to a CharField. However, it can't be larger than 500 characters, which may be a limitation in case an answer to a question is larger than 500 characters. I guess if this happens, an issue will be raised !

Thanks for the debugging tip @c-w

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
v1.6.0
  
To do
Development

Successfully merging this pull request may close these issues.

None yet

3 participants