You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of MLRun CE.
Reproducible Example
importmlrunimportmlrun.feature_storeasfstorefrommlrun.datastore.targetsimportRedisNoSqlTarget,SQLTargetfrommlrun.data_types.data_typesimportValueTypeimportpandasaspddefmysql_test(project_name="bb"):
mlrun.set_env_from_file("mlrun-nonprod.env")
project=mlrun.get_or_create_project(project_name, context='./', user_project=False)
feature_name="basic-party"feature_set=fstore.FeatureSet(feature_name, entities=[fstore.Entity("party-id", value_type=ValueType.INT64)],
engine="storey")
feature_set.add_feature(fstore.Feature(name="party-type"))
conn="mysql+pymysql://jist:jist@localhost:3306/test"# Sample 1. - ingest the FeatureSet works fine with schema definitionfeature_set.set_targets(targets=[SQLTarget(name="we2", db_url=conn, table_name='my_table',
schema= {'party-id': int, 'party-type': str},
create_table=True,
primary_key_column='party-id')],
with_defaults=False)
# Sample 2. - Error (without schema definition)# File "C:\Python\test\.venv\lib\site-packages\mlrun\datastore\targets.py", line 294, in add_target_steps# table = driver.get_table_object() or table# File "C:\Python\test\.venv\lib\site-packages\mlrun\datastore\targets.py", line 1690, in get_table_object# (db_path, table_name, _, _, primary_key, _) = self._parse_url()# File "C:\Python\test\.venv\lib\site-packages\mlrun\datastore\targets.py", line 1804, in _parse_url# path = self.path[len("mlrunSql:///") :]# TypeError: 'NoneType' object is not subscriptablefeature_set.set_targets(targets=[SQLTarget(name="we3",
db_url=conn)],
with_defaults=False)
# Sample 3. - Error (without schema definition)# File "C:\Python\test\.venv\lib\site-packages\pymysql\protocol.py", line 221, in raise_for_error# err.raise_mysql_exception(self._data)# File "C:\Python\test\.venv\lib\site-packages\pymysql\err.py", line 143, in raise_mysql_exception# raise errorclass(errno, errval)# sqlalchemy.exc.ProgrammingError: (pymysql.err.ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')' at line 2")# [SQL:# CREATE TABLE my_table2 (# )## ]# (Background on this error at: https://sqlalche.me/e/14/f405)feature_set.set_targets(targets=[SQLTarget(name="we4",
db_url=conn,
table_name='my_table2',
create_table=True)],
with_defaults=False)
feature_set.save()
data={"party-id": [1,2,3],
"party-type": ["a1", "b2", "c3"]}
dataFrm=pd.DataFrame(data)
fstore.ingest(feature_set,dataFrm,overwrite=True)
if__name__=='__main__':
mysql_test(project)
Issue Description
I mentioned in sample three cases. First is fine and it works correctly, second generated error ' TypeError: 'NoneType' object is not subscriptable' and third generated error 'You have an error in your SQL syntax; check ...'.
It seems that SQLTarget support only manual mapping of FeatureSet to SQLTarget based on parameter schema and it is different behavioral than in e.g. RedisNoSqlTarget where the redis target works without schema mapping).
Expected Behavior
It will be very useful to support SQLTarget without manual schema definition, I think about ability to use only this easy sample SQLTarget(name="we3", db_url=conn)
It means, the schema will be generate automatically (in background) based on FeatureSet definition (in case, that schema was not added manually).
The text was updated successfully, but these errors were encountered:
george0st
changed the title
[Bug]: SQLTarget - missing automapping to FeatureSet definition
[Bug]: SQLTarget - missing automapping for FeatureSet definition
Feb 2, 2024
MLRun Version checks
I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of MLRun CE.
Reproducible Example
Issue Description
I mentioned in sample three cases. First is fine and it works correctly, second generated error ' TypeError: 'NoneType' object is not subscriptable' and third generated error 'You have an error in your SQL syntax; check ...'.
It seems that SQLTarget support only manual mapping of FeatureSet to SQLTarget based on parameter schema and it is different behavioral than in e.g. RedisNoSqlTarget where the redis target works without schema mapping).
Expected Behavior
It will be very useful to support SQLTarget without manual schema definition, I think about ability to use only this easy sample
SQLTarget(name="we3", db_url=conn)
It means, the schema will be generate automatically (in background) based on FeatureSet definition (in case, that schema was not added manually).
Installation OS
Windows
Installation Method
Docker
Python Version
3.9.10
MLRun Version
1.5.2
Additional Information
Setting for MySQL see https://github.com/george0st/qgate-sln-mlrun/blob/master/docs/mysql.md
The text was updated successfully, but these errors were encountered: