Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot import data using neo4j-etl export #69

Open
AnoYuan opened this issue Apr 3, 2020 · 1 comment
Open

Cannot import data using neo4j-etl export #69

AnoYuan opened this issue Apr 3, 2020 · 1 comment

Comments

@AnoYuan
Copy link

AnoYuan commented Apr 3, 2020

My process is like this:
1.generate mapping.json
bash ./neo4j-etl generate-metadata-mapping --rdbms:url jdbc:mysql://mysql-ip:mysql-port --rdbms:user root --rdbms:password root --output-mapping-file "/var/lib/neo4j/mapping/test.json" --rdbms:schema test
2.Import data
bash ./neo4j-etl export --rdbms:url jdbc:mysql://mysql-ip:mysql-port/test --rdbms:user root --rdbms:password root --destination /var/lib/neo4j/data/databases/test.db --import-tool /var/lib/neo4j/bin/ --csv-directory /var/lib/neo4j/import --neo4j:url bolt://neo4j-ip:7687 --neo4j:user neo4j --neo4j:password 123456 --options-file /var/lib/neo4j/conf/options.json --force --mapping-file "/var/lib/neo4j/mapping/test.json" --debug

logs:
INFO: Reading options from file /root/docker_data/neo4j/conf/options.json.
FINE: Connecting to database...
FINE: Connected to database
INFO: Reading metadata mapping from file: /root/docker_data/neo4j/mapping/test.json
FINE: Using java.util.logging as the default logging framework
FINE: -Dio.netty.eventLoopThreads: 16
FINE: -Dio.netty.threadLocalMap.stringBuilder.initialSize: 1024
FINE: -Dio.netty.threadLocalMap.stringBuilder.maxSize: 4096
FINE: -Dio.netty.noKeySetOptimization: false
FINE: -Dio.netty.selectorAutoRebuildThreshold: 512
FINE: -Dio.netty.noUnsafe: false
FINE: Java version: 8
FINE: sun.misc.Unsafe.theUnsafe: available
FINE: sun.misc.Unsafe.copyMemory: available
FINE: java.nio.Buffer.address: available
FINE: direct buffer constructor: available
FINE: java.nio.Bits.unaligned: available, true
FINE: jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable prior to Java9
FINE: java.nio.DirectByteBuffer.(long, int): available
FINE: sun.misc.Unsafe: available
FINE: -Dio.netty.tmpdir: /tmp (java.io.tmpdir)
FINE: -Dio.netty.bitMode: 64 (sun.arch.data.model)
FINE: -Dio.netty.maxDirectMemory: 7495745536 bytes
FINE: -Dio.netty.uninitializedArrayAllocationThreshold: -1
FINE: java.nio.ByteBuffer.cleaner(): available
FINE: -Dio.netty.noPreferDirect: false
FINE: org.jctools-core.MpscChunkedArrayQueue: available
FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@7bba5817
FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@350aac89
FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@1c5920df
FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@17f9d882
FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@79e4c792
FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@196a42c3
FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@4c60d6e9
FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@15043a2f
FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@4a83a74a
FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@1349883
FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@4b29d1d2
FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@7f485fda
FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@28261e8e
FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@d737b89
FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@8519cb4
FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@35dab4eb
INFO: Direct driver instance 1485089044 created for server address ip:port
INFO: Running ETL on Neo4j 3.5.5 - COMMUNITY
INFO: Exporting from RDBMS to CSV...
INFO: CSV directory: /root/docker_data/neo4j/import/csv-012
FINE: Executing command 'chmod 0777 /root/docker_data/neo4j/import/csv-012'
FINER: Command finished [Command: 'chmod 0777 /root/docker_data/neo4j/import/csv-012', CommandResult { ExitValue: 0, Stdout: '', Stderr: '', DurationMillis: 9 }]
FINE: Connecting to database...
FINE: Connected to database
INFO: Writing CSV headers for node NODE_test.ip_95b7044d-0ef9-4c6c-92cf-cfb0e09c7d66
FINEST: SELECT test.ip.id AS id, test.ip.id AS id, test.ip.ip AS ip, "Ip" AS _NODE_LABEL_ FROM test.ip
Export time: 0.03 (s)
INFO: Creating Neo4j store from CSV...
WARNING: Neo4j is running! You can run neo4j-import tool only if the database is offline
INFO: /root/docker_data/neo4j/data/databases/test.db

WARNING: Neo4j is running! You can run neo4j-import tool only if the database is offline
WARNING: Neo4j is running! You can run neo4j-import tool only if the database is offline
WARNING: Neo4j is running! You can run neo4j-import tool only if the database is offline
WARNING: Neo4j is running! You can run neo4j-import tool only if the database is offline

What should I do?

@jexp
Copy link
Member

jexp commented Jun 22, 2020

Can you stop your neo4j database before you start your ETL process?
Or use the online import mode.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants