Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mongodump/mongorestore 报错 #211

Open
mervynzhang opened this issue Nov 14, 2023 · 16 comments
Open

mongodump/mongorestore 报错 #211

mervynzhang opened this issue Nov 14, 2023 · 16 comments

Comments

@mervynzhang
Copy link

mervynzhang commented Nov 14, 2023 •
edited

之前用apt install的mongodump/mongorestore， 版本有点老， 下载了https://fastdl.mongodb.org/tools/db/mongodb-database-tools-ubuntu2004-x86_64-100.9.1.deb，

mongodump 可以用了，但导出的collection 比mongodb 多很多。

mongodb

mongodump --db=authordb --archive=authordb.bson
2023-11-14T15:42:29.728+0800    writing authordb.authors_restore to archive 'authordb.bson'
2023-11-14T15:42:29.734+0800    writing authordb.authors to archive 'authordb.bson'
2023-11-14T15:42:29.735+0800    done dumping authordb.authors_restore (2 documents)
2023-11-14T15:42:29.737+0800    done dumping authordb.authors (4 documents)

lealone

mongodump --db=authordb --archive=authordb2.bson
2023-11-14T15:44:48.291+0800    writing authordb.POSTS to archive 'authordb2.bson'
2023-11-14T15:44:48.293+0800    writing authordb.COLUMN_PRIVILEGES to archive 'authordb2.bson'
2023-11-14T15:44:48.293+0800    writing authordb.AUTHORS_RESTORE to archive 'authordb2.bson'
2023-11-14T15:44:48.298+0800    writing authordb.AUTHORS to archive 'authordb2.bson'
2023-11-14T15:44:48.301+0800    done dumping authordb.COLUMN_PRIVILEGES (0 documents)
2023-11-14T15:44:48.302+0800    done dumping authordb.POSTS (1 document)
2023-11-14T15:44:48.302+0800    done dumping authordb.AUTHORS_RESTORE (2 documents)
2023-11-14T15:44:48.302+0800    writing authordb.SETTINGS to archive 'authordb2.bson'
2023-11-14T15:44:48.303+0800    writing authordb.HELP to archive 'authordb2.bson'
2023-11-14T15:44:48.303+0800    writing authordb.RIGHTS to archive 'authordb2.bson'
2023-11-14T15:44:48.308+0800    done dumping authordb.AUTHORS (3 documents)
2023-11-14T15:44:48.310+0800    writing authordb.TRIGGERS to archive 'authordb2.bson'
2023-11-14T15:44:48.316+0800    done dumping authordb.SETTINGS (0 documents)
2023-11-14T15:44:48.317+0800    writing authordb.CATALOGS to archive 'authordb2.bson'
2023-11-14T15:44:48.318+0800    done dumping authordb.RIGHTS (0 documents)
2023-11-14T15:44:48.319+0800    writing authordb.CROSS_REFERENCES to archive 'authordb2.bson'
2023-11-14T15:44:48.320+0800    done dumping authordb.HELP (0 documents)
2023-11-14T15:44:48.321+0800    done dumping authordb.TRIGGERS (0 documents)
2023-11-14T15:44:48.321+0800    done dumping authordb.CROSS_REFERENCES (0 documents)
2023-11-14T15:44:48.321+0800    writing authordb.FUNCTION_ALIASES to archive 'authordb2.bson'
2023-11-14T15:44:48.321+0800    writing authordb.SESSION_STATE to archive 'authordb2.bson'
2023-11-14T15:44:48.323+0800    writing authordb.VIEWS to archive 'authordb2.bson'
2023-11-14T15:44:48.324+0800    done dumping authordb.FUNCTION_ALIASES (0 documents)
2023-11-14T15:44:48.327+0800    writing authordb.TYPE_INFO to archive 'authordb2.bson'
2023-11-14T15:44:48.328+0800    done dumping authordb.SESSION_STATE (0 documents)
2023-11-14T15:44:48.328+0800    done dumping authordb.CATALOGS (0 documents)
2023-11-14T15:44:48.328+0800    writing authordb.CONSTRAINTS to archive 'authordb2.bson'
2023-11-14T15:44:48.329+0800    writing authordb.COLUMNS to archive 'authordb2.bson'
2023-11-14T15:44:48.333+0800    done dumping authordb.VIEWS (0 documents)
2023-11-14T15:44:48.336+0800    writing authordb.DB_OBJECTS to archive 'authordb2.bson'
2023-11-14T15:44:48.337+0800    done dumping authordb.TYPE_INFO (0 documents)
2023-11-14T15:44:48.339+0800    writing authordb.DOMAINS to archive 'authordb2.bson'
2023-11-14T15:44:48.339+0800    done dumping authordb.COLUMNS (0 documents)
2023-11-14T15:44:48.339+0800    done dumping authordb.CONSTRAINTS (0 documents)
2023-11-14T15:44:48.340+0800    writing authordb.SERVICES to archive 'authordb2.bson'
2023-11-14T15:44:48.341+0800    done dumping authordb.DB_OBJECTS (0 documents)
2023-11-14T15:44:48.342+0800    writing authordb.FUNCTION_COLUMNS to archive 'authordb2.bson'
2023-11-14T15:44:48.342+0800    writing authordb.SEQUENCES to archive 'authordb2.bson'
2023-11-14T15:44:48.350+0800    done dumping authordb.DOMAINS (0 documents)
2023-11-14T15:44:48.351+0800    writing authordb.LOCKS to archive 'authordb2.bson'
2023-11-14T15:44:48.351+0800    done dumping authordb.SERVICES (0 documents)
2023-11-14T15:44:48.352+0800    done dumping authordb.SEQUENCES (0 documents)
2023-11-14T15:44:48.352+0800    writing authordb.TABLE_PRIVILEGES to archive 'authordb2.bson'
2023-11-14T15:44:48.355+0800    writing authordb.DATABASES to archive 'authordb2.bson'
2023-11-14T15:44:48.356+0800    done dumping authordb.FUNCTION_COLUMNS (0 documents)
2023-11-14T15:44:48.356+0800    done dumping authordb.LOCKS (0 documents)
2023-11-14T15:44:48.357+0800    writing authordb.TABLE_TYPES to archive 'authordb2.bson'
2023-11-14T15:44:48.357+0800    writing authordb.TABLES to archive 'authordb2.bson'
2023-11-14T15:44:48.358+0800    done dumping authordb.TABLE_PRIVILEGES (0 documents)
2023-11-14T15:44:48.360+0800    writing authordb.PLUGINS to archive 'authordb2.bson'
2023-11-14T15:44:48.363+0800    done dumping authordb.DATABASES (0 documents)
2023-11-14T15:44:48.364+0800    writing authordb.ROLES to archive 'authordb2.bson'
2023-11-14T15:44:48.364+0800    done dumping authordb.TABLES (0 documents)
2023-11-14T15:44:48.364+0800    done dumping authordb.TABLE_TYPES (0 documents)
2023-11-14T15:44:48.365+0800    writing authordb.SESSIONS to archive 'authordb2.bson'
2023-11-14T15:44:48.368+0800    writing authordb.USERS to archive 'authordb2.bson'
2023-11-14T15:44:48.369+0800    done dumping authordb.PLUGINS (0 documents)
2023-11-14T15:44:48.370+0800    writing authordb.COLLATIONS to archive 'authordb2.bson'
2023-11-14T15:44:48.373+0800    done dumping authordb.SESSIONS (0 documents)
2023-11-14T15:44:48.375+0800    done dumping authordb.USERS (0 documents)
2023-11-14T15:44:48.375+0800    done dumping authordb.COLLATIONS (0 documents)
2023-11-14T15:44:48.375+0800    done dumping authordb.ROLES (0 documents)
2023-11-14T15:44:48.375+0800    writing authordb.CONSTANTS to archive 'authordb2.bson'
2023-11-14T15:44:48.376+0800    writing authordb.SCHEMAS to archive 'authordb2.bson'
2023-11-14T15:44:48.376+0800    writing authordb.TABLE_ALTER_HISTORY to archive 'authordb2.bson'
2023-11-14T15:44:48.377+0800    writing authordb.INDEXES to archive 'authordb2.bson'
2023-11-14T15:44:48.386+0800    done dumping authordb.SCHEMAS (0 documents)
2023-11-14T15:44:48.386+0800    done dumping authordb.TABLE_ALTER_HISTORY (0 documents)
2023-11-14T15:44:48.386+0800    done dumping authordb.INDEXES (0 documents)
2023-11-14T15:44:48.386+0800    writing authordb.QUERY_STATISTICS to archive 'authordb2.bson'
2023-11-14T15:44:48.388+0800    done dumping authordb.CONSTANTS (0 documents)
2023-11-14T15:44:48.390+0800    done dumping authordb.QUERY_STATISTICS (0 documents)
@mervynzhang mervynzhang changed the title mongodump/mongorestore 报错 Connection reset by peer mongodump/mongorestore 报错 Nov 14, 2023
@mervynzhang
Copy link
Author

mongorestore lealone 还有报错

mongorestore --drop --archive=authordb.bson
2023-11-14T15:43:31.890+0800 error getting server version: buildInfo.versionArray had fewer than 3 elements

@codefollower
Copy link
Member

我刚装了 mongodump 试了一下是可以的。
如是出现上面的 WARN 只是 lealone 发现客户端把连接关闭了而已。
那些表是 lealone 内部的。

@codefollower
Copy link
Member

mongorestore lealone 还有报错

mongorestore --drop --archive=authordb.bson 2023-11-14T15:43:31.890+0800 error getting server version: buildInfo.versionArray had fewer than 3 elements

buildInfo 命令还没有实现全部，给客户端的信息它要求太严所以解析错误了。

@mervynzhang
Copy link
Author

给buildInfo 加了versionArray 能跑到下一步了， 但有新的报错， 看起来像bug

        case "buildInfo": {
            BsonDocument document = new BsonDocument();
            BsonArray versionArray = new BsonArray();
            versionArray.add(new BsonInt32(6));
            versionArray.add(new BsonInt32(0));
            versionArray.add(new BsonInt32(0));
            append(document, "version", "6.0.0");
            document.append("versionArray", versionArray);

现在报错

17:11:25.423 [ScheduleService-3] ERROR org.lealone.plugins.mongo.server.MongoTask - Failed to execute command: insert
org.lealone.common.exceptions.JdbcSQLException: Column "_ID" not found [42122-0]
        at org.lealone.common.exceptions.DbException.getJdbcSQLException(DbException.java:342) ~[classes/:?]
        at org.lealone.common.exceptions.DbException.get(DbException.java:173) ~[classes/:?]
        at org.lealone.common.exceptions.DbException.get(DbException.java:150) ~[classes/:?]
        at org.lealone.db.table.Table.getColumn(Table.java:614) ~[classes/:?]
        at org.lealone.plugins.mongo.bson.BsonBase.parseColumn(BsonBase.java:181) ~[classes/:?]
        at org.lealone.plugins.mongo.bson.command.BCInsert.addRows(BCInsert.java:52) ~[classes/:?]
        at org.lealone.plugins.mongo.bson.command.BCInsert.execute(BCInsert.java:33) ~[classes/:?]
        at org.lealone.plugins.mongo.bson.command.BsonCommand.execute(BsonCommand.java:261) ~[classes/:?]
        at org.lealone.plugins.mongo.server.MongoTask.run(MongoTask.java:44) [classes/:?]
        at org.lealone.server.SessionInfo.runTask(SessionInfo.java:106) [classes/:?]
        at org.lealone.server.SessionInfo.submitTask(SessionInfo.java:69) [classes/:?]
        at org.lealone.plugins.mongo.server.MongoServerConnection.handleCommand(MongoServerConnection.java:249) [classes/:?]
        at org.lealone.plugins.mongo.server.MongoServerConnection.handleMessage(MongoServerConnection.java:231) [classes/:?]
        at org.lealone.plugins.mongo.server.MongoServerConnection.handle(MongoServerConnection.java:185) [classes/:?]
        at org.lealone.net.nio.NioEventLoop.read(NioEventLoop.java:276) [classes/:?]
        at org.lealone.net.nio.NioEventLoop.handleSelectedKeys(NioEventLoop.java:493) [classes/:?]
        at org.lealone.server.GlobalScheduler.runEventLoop(GlobalScheduler.java:395) [classes/:?]
        at org.lealone.server.GlobalScheduler.run(GlobalScheduler.java:100) [classes/:?]
        at java.lang.Thread.run(Thread.java:829) [?:?]

@codefollower
Copy link
Member

更新一下代码试试

@mervynzhang
Copy link
Author

已经是最新代码了。 我debug了下， 问题应该是这里

pymongo insert 新表的话， 这里table == null
mongorestore的话， 这里 table = StandardTable@30 "PUBLIC.AUTHORS_RESTORE", columnMap 是空的， 所以会报“_id” 为空

@codefollower
Copy link
Member

如果 mongorestore 触发的是 create 命令，且 viewOn 参数没有指定， lealone 实现这个 create 命令就是建一张空表，
后续 mongorestore 再往这张表 insert 数据时 lealone 目前还没有实现动态增加字段的功能，所以会报告字段找不到。

@mervynzhang
Copy link
Author

如果 mongorestore 触发的是 create 命令，且 viewOn 参数没有指定， lealone 实现这个 create 命令就是建一张空表， 后续 mongorestore 再往这张表 insert 数据时 lealone 目前还没有实现动态增加字段的功能，所以会报告字段找不到。

这样的话，虽然是mongo的api， 但还是需要用固定的schema。 原来的字段是TIMESTAMP， 后边document改成string就报错了。 这样的话，数据基本没法导入了，原有数据里大量的同一个字段不同类型的情况。

pymongo.errors.OperationFailure: Cannot parse "TIMESTAMP" constant "date ya" [22007-0], full error: {'minWireVersion': 0, 'maxWireVersion': 17, 'ok': 0, 'n': 1, 'code': 22007, 'errmsg': 'Cannot parse "TIMESTAMP" constant "date ya" [22007-0]'}

@codefollower
Copy link
Member

如果 mongorestore 触发的是 create 命令，且 viewOn 参数没有指定， lealone 实现这个 create 命令就是建一张空表， 后续 mongorestore 再往这张表 insert 数据时 lealone 目前还没有实现动态增加字段的功能，所以会报告字段找不到。

这样的话，虽然是mongo的api， 但还是需要用固定的schema。 原来的字段是TIMESTAMP， 后边document改成string就报错了。 这样的话，数据基本没法导入了，原有数据里大量的同一个字段不同类型的情况。

pymongo.errors.OperationFailure: Cannot parse "TIMESTAMP" constant "date ya" [22007-0], full error: {'minWireVersion': 0, 'maxWireVersion': 17, 'ok': 0, 'n': 1, 'code': 22007, 'errmsg': 'Cannot parse "TIMESTAMP" constant "date ya" [22007-0]'}

这种动态功能容易实现，分析当前执行的文档字段个数和类型跟现有表结构比较一下，动作执行 alter 语句修改表结构就好了，都是体力活，只是优先级排在后面，现在忙着别的事，工期还没有排到 mongodb 插件。

mongodb、mysql、postgresql 三个内置插件依然是开发阶段，没有到生产级别的，不支持的特性后续补上就好了。
如果你有兴趣，也很欢迎你提交 PR。

@codefollower
Copy link
Member

codefollower commented Nov 22, 2023 •
edited

如果 mongorestore 触发的是 create 命令，且 viewOn 参数没有指定， lealone 实现这个 create 命令就是建一张空表， 后续 mongorestore 再往这张表 insert 数据时 lealone 目前还没有实现动态增加字段的功能，所以会报告字段找不到。

这样的话，虽然是mongo的api， 但还是需要用固定的schema。 原来的字段是TIMESTAMP， 后边document改成string就报错了。 这样的话，数据基本没法导入了，原有数据里大量的同一个字段不同类型的情况。

pymongo.errors.OperationFailure: Cannot parse "TIMESTAMP" constant "date ya" [22007-0], full error: {'minWireVersion': 0, 'maxWireVersion': 17, 'ok': 0, 'n': 1, 'code': 22007, 'errmsg': 'Cannot parse "TIMESTAMP" constant "date ya" [22007-0]'}

这种动态功能容易实现，分析当前执行的文档字段个数和类型跟现有表结构比较一下，动态执行 alter 语句修改表结构就好了，都是体力活，只是优先级排在后面，现在忙着别的事，工期还没有排到 mongodb 插件。

一个字段想兼容多种类型无非是全转到字符串和二进制。

mongodb、mysql、postgresql 三个内置插件依然是开发阶段，没有到生产级别的，不支持的特性后续补上就好了。
如果你有兴趣，也很欢迎你提交 PR。

codefollower added a commit that referenced this issue Dec 5, 2023
@codefollower
mongo: 支持动态增加字段、修改字段类型 #211
02d6bc6
@mervynzhang
Copy link
Author

我主要做devopps，开发经验太少， 能读代码， 写新功能代码就有点力不从心了。

拉最新代码测试了下， 少量数据可以mongorestore了。 大量导入就报 NullPointerException

ERROR [FsyncService] [2023-12-07T10:12:08,766] Failed to sync redo log
java.lang.NullPointerException: null
	at org.lealone.transaction.aote.log.RedoLogChunk.checkpoint(RedoLogChunk.java:268) ~[lealone-6.0.0-SNAPSHOT.jar:?]
	at org.lealone.transaction.aote.log.RedoLogChunk.save(RedoLogChunk.java:170) ~[lealone-6.0.0-SNAPSHOT.jar:?]
	at org.lealone.transaction.aote.log.RedoLog.save(RedoLog.java:123) ~[lealone-6.0.0-SNAPSHOT.jar:?]
	at org.lealone.transaction.aote.log.LogSyncService.sync(LogSyncService.java:112) [lealone-6.0.0-SNAPSHOT.jar:?]
	at org.lealone.transaction.aote.log.LogSyncService.run(LogSyncService.java:88) [lealone-6.0.0-SNAPSHOT.jar:?]
	at org.lealone.main.Lealone.run(Lealone.java:211) [lealone-6.0.0-SNAPSHOT.jar:?]
	at org.lealone.main.Lealone.start(Lealone.java:135) [lealone-6.0.0-SNAPSHOT.jar:?]
	at org.lealone.main.Lealone.start(Lealone.java:97) [lealone-6.0.0-SNAPSHOT.jar:?]
	at org.lealone.main.Lealone.main(Lealone.java:51) [lealone-6.0.0-SNAPSHOT.jar:?]

image

@codefollower
Copy link
Member

checkpoint 还有 bug，还未修复

codefollower added a commit that referenced this issue Dec 7, 2023
@codefollower
前一个检查点没有执行完时不能执行第二个检查点，否则会导致各种异常 #211
0da6ec8
@mervynzhang
Copy link
Author

报告一个新错误， 还是mongorestore 出来的

ERROR 16:54:38.628 Failed to execute command: insert
org.lealone.common.exceptions.JdbcSQLException: Column count does not match [21002-0]
        at org.lealone.common.exceptions.DbException.getJdbcSQLException(DbException.java:342) ~[lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.common.exceptions.DbException.get(DbException.java:173) ~[lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.common.exceptions.DbException.get(DbException.java:150) ~[lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.common.exceptions.DbException.get(DbException.java:139) ~[lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.sql.dml.MerSert.prepare(MerSert.java:116) ~[lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.plugins.mongo.bson.command.BCInsert.addRows(BCInsert.java:83) ~[lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.plugins.mongo.bson.command.BCInsert.execute(BCInsert.java:36) ~[lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.plugins.mongo.bson.command.BsonCommand.execute(BsonCommand.java:262) ~[lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.plugins.mongo.server.MongoTask.run(MongoTask.java:44) [lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.server.scheduler.SessionInfo.runTask(SessionInfo.java:104) [lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.server.scheduler.SessionInfo.submitTask(SessionInfo.java:68) [lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.plugins.mongo.server.MongoServerConnection.handleCommand(MongoServerConnection.java:236) [lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.plugins.mongo.server.MongoServerConnection.handleMessage(MongoServerConnection.java:218) [lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.plugins.mongo.server.MongoServerConnection.handle(MongoServerConnection.java:172) [lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.net.nio.NioEventLoop.read(NioEventLoop.java:275) [lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.net.nio.NioEventLoop.handleSelectedKeys(NioEventLoop.java:492) [lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.net.NetScheduler.runEventLoop(NetScheduler.java:62) [lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.server.scheduler.GlobalScheduler.run(GlobalScheduler.java:70) [lealone-6.0.0-SNAPSHOT.jar:?]
        at java.lang.Thread.run(Thread.java:829) [?:?]

@mervynzhang
Copy link
Author

mongorestore 还有一个错，但lealone 没有报错

Hexadecimal string with odd number of characters: "xyz" [90003-0]

codefollower added a commit that referenced this issue Dec 11, 2023
@codefollower
mongo:执行批量insert命令时需要处理记录的字段个数不一致或字段顺序不同的情况 #211
89247e3
@mervynzhang
Copy link
Author

有新报错

ERROR 11:18:08.343 Failed to sync redo log
java.lang.NullPointerException: null
        at org.lealone.storage.fs.FileStorage.sync(FileStorage.java:436) ~[lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.transaction.aote.log.RedoLogChunk.save(RedoLogChunk.java:119) ~[lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.transaction.aote.log.RedoLog.save(RedoLog.java:123) ~[lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.transaction.aote.log.LogSyncService.sync(LogSyncService.java:112) [lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.transaction.aote.log.LogSyncService.run(LogSyncService.java:88) [lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.main.Lealone.run(Lealone.java:211) [lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.main.Lealone.start(Lealone.java:135) [lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.main.Lealone.start(Lealone.java:97) [lealone-6.0.0-SNAPSHOT.jar:?]
        at org.lealone.main.Lealone.main(Lealone.java:51) [lealone-6.0.0-SNAPSHOT.jar:?]
Exception in thread "ScheduleService-5" java.lang.IllegalStateException: Chunk too large, max size: 2147475455, current size: 2147530719 [6.0.0/2]
        at org.lealone.common.util.DataUtils.newIllegalStateException(DataUtils.java:616)
        at org.lealone.storage.aose.btree.page.Page.updateChunkAndPage(Page.java:353)
        at org.lealone.storage.aose.btree.page.LeafPage.writeRowStorage(LeafPage.java:251)
        at org.lealone.storage.aose.btree.page.LeafPage.writeUnsavedRecursive(LeafPage.java:226)
        at org.lealone.storage.aose.btree.page.NodePage.writeUnsavedRecursive(NodePage.java:205)
        at org.lealone.storage.aose.btree.page.NodePage.writeUnsavedRecursive(NodePage.java:205)
        at org.lealone.storage.aose.btree.page.NodePage.writeUnsavedRecursive(NodePage.java:205)
        at org.lealone.storage.aose.btree.page.NodePage.writeUnsavedRecursive(NodePage.java:205)
        at org.lealone.storage.aose.btree.BTreeStorage.executeSave(BTreeStorage.java:365)
        at org.lealone.storage.aose.btree.BTreeStorage.save(BTreeStorage.java:336)
        at org.lealone.storage.aose.btree.BTreeMap.fullGc(BTreeMap.java:429)
        at org.lealone.transaction.aote.CheckpointService.fullGc(CheckpointService.java:161)
        at org.lealone.transaction.aote.AOTransactionEngine.fullGc(AOTransactionEngine.java:72)
        at org.lealone.server.scheduler.GlobalScheduler.gc(GlobalScheduler.java:210)
        at org.lealone.server.scheduler.GlobalScheduler.executeNextStatement(GlobalScheduler.java:223)
        at org.lealone.server.scheduler.GlobalScheduler.run(GlobalScheduler.java:68)
        at java.base/java.lang.Thread.run(Thread.java:829)

mongorestore的报错

2023-12-11T11:19:41.486+0800    finished restoring test.test (282053 documents, 0 failures)
2023-12-11T11:19:41.486+0800    Failed: test.test: error restoring from archive on stdin: General error: "java.lang.IllegalStateException: This map is closed [6.0.0/4]" [50000-0]
2023-12-11T11:19:41.486+0800    282053 document(s) restored successfully. 0 document(s) failed to restore

@codefollower
Copy link
Member

java.lang.IllegalStateException: Chunk too large, max size: 2147475455, current size: 2147530719
这个异常是没有及时刷脏页，导致 chunk 的大小超过2G了。这个问题先放着，后续再处理。

devqm pushed a commit to devqm/Lealone that referenced this issue May 14, 2024
@codefollower
mongo: 支持动态增加字段、修改字段类型 lealone#211
b1f8a46
devqm pushed a commit to devqm/Lealone that referenced this issue May 14, 2024
@codefollower
前一个检查点没有执行完时不能执行第二个检查点，否则会导致各种异常 lealone#211
b82c187
devqm pushed a commit to devqm/Lealone that referenced this issue May 14, 2024
@codefollower
mongo:执行批量insert命令时需要处理记录的字段个数不一致或字段顺序不同的情况 lealone#211
1b603b1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@codefollower @mervynzhang