Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get weights and biases from trained model #5655

Open
dsilkersahin opened this issue Aug 8, 2023 · 0 comments
Open

Get weights and biases from trained model #5655

dsilkersahin opened this issue Aug 8, 2023 · 0 comments

Comments

@dsilkersahin
Copy link

dsilkersahin commented Aug 8, 2023

Sparkling Water Version

3.42.0.2-1-2.4

Issue description

In order to do online learning, we need to get weights and biases from trained model. Although parameter .setExportWeightsAndBiases( ) is set to true, I am never able to access the weights matrix or other weights and biases details. (just descriptive stats results exist about weights)

When training the model, this package is used:
import ai.h2o.sparkling.ml.algos.{H2ODeepLearning,}

First Training

val dl = new H2ODeepLearning()
  .setLabelCol("*****")
  .setFeaturesCol("*****")
  .setConvertUnknownCategoricalLevelsToNa(true)
  .setExportWeightsAndBiases(true)

  val firstModel= dl.fit(train_df.repartition(count.toInt + 10))

When loading the model to re-fit again by using set initial weights, this package is used:
import ai.h2o.sparkling.ml.models.{H2ODeepLearningMOJOModel

UpadteTraining

val updated_dl= new H2ODeepLearning()
  .setLabelCol("****")
  .setFeaturesCol("****")
  .setConvertUnknownCategoricalLevelsToNa(true)
  .setInitialWeights(**firstModel.weights**) ----->(firstModel.weights does not exist)

updated_dl.fit(batch_df).......

However The model that is loaded as mojomodel has not weights or biases details.

I look forward to your help.
Regards,

Programming language used

Scala

Programming language version

2.11

What environment are you running Sparkling Water on?

Cloud Managed Spark (like Databricks, AWS Glue)

Environment version info

cdh6.3.2

Brief cluster specification

Relevant log output

-

Code to reproduce the issue

-
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant