Skip to content

Commit

Permalink
Version 3.6
Browse files Browse the repository at this point in the history
Major Data Updates:
- New monster column: "Size" and "Type" (multi-values).
- New companion column: "Size" and "Type" (multi-values).
- New item Category "Lair".  Reclassify buildings into this category.
- New item Type "Instruments", "Implement", and "Lighting" (wondrous only).
- New item rarity: "Mundane", "Pact weapon", "Masterwork", and "Unique".
- Move "Artifact" item class to Rarity column.
- Reclassify "Alchemical Item" into "Consumable" Category.
- Class "Abilities" now list only primary abilities (excludes secondary abilities).
- Add glossary0453 (Item Set).
- Change glossary381 (Concordance) to Artifact by adding relevant rules.

New Viewer Features:
- Swipe left/right to flip page (on browsers that do not consume swipe gestures, i.e. non-uc browser).
- Lookup or search from context menu.  Take selected text, or right-clicked text if selection is empty.
- Add filter / reset filter from context menu.  Take right-clicked cell in search result.
- Reduce data file count to 463 (from 1800+) for faster file copy and better data consolidation.
- Search by item id (single id only).
- Use "NIL" (without quotes) in filter to show results that are empty in that column, or "-NIL" to exclude empties.
- Show quick actions (switch to full text / all category / clear search / clear filter) when a search yield no result.
- Add web app manifest.
- Smaller html size (load faster by moving embedded images, css, and js to subfolder).
- Improved error handling.  Add message for Android apps.
- Elimination of dynamic path detection and correction, less error prone.

New Downloader Features:
- Project page and download help now links to subscription purchase page.
- Add optional LZMA + Base85 data compression, default off because both compress and decompress can be slow.  Auto-off if heap < 512mb.
- Add thread count control.
- Add fix data checkbox.  Export and dumps now works on both fixed and unfixed data.
- Export and dump can be initiated when data is being loaded, instead of after all data is loaded.
- Improves efficiency of main export (use less memory, faster text escape).
- Start up faster (crawler webview is now created on-demand).

Viewer Bug Fixes:
- Text selection by mouse no longer triggers lookup.
- Lookup no longer pops when clicked in an empty area.
- Result list no longer jump on right click, and will open new tab on Ctrl click.
- Pressing left/right arrow in search/filter box no longer flip result.
- Lookup no longer match partial-words.
- Re-detect language (forget previous preference) when switching back to auto.
- Default to English instead of Auto.  Auto or Chinese must be manually requested.
- Switching language will now update current search list.
- Localised all filter fields.
- Help now includes number range syntax and number unit.
- Fix wrong search when initiated from url (e.g. browser session load).
- Fix number-only search/filter not working correctly.
- Fix scroll to top button not working in Firefox.
- Fix page count may show 1/0 on no result.
- Update soft keyboard detection for new browsers.
- Fix category link position in data entry.
- Fix power title text overlap on mobile.
- Switch column width to dynamic on wide screen.
- Non-result rows no longer highlight on hover.
- Fix lookup of names with é or û (Ogrémoch, Selûne, etc.)
- Fix lookup of "Dying".
- Fix lookup of item171 (belt pouch).
- "Please allow javascript" pointer position now response to screen size.
- Data css improvements.

Downloader Bug Fixes:
- Detect and report Out of Memory error (as best as we can anyway).
- Fix export synchronisation; should see much less or no errors when exporting / dumping on multi-core systems.
- Refactor data structure; fixed data is now independent from unfixed data and in same format.
- Add auto-generated test cases on fixes to detect regression.  Only run when debug log is on.

Downloader Data Fix:
- Always put level/type in entry title after name of the entry. (Affects layout)
- Default monster size to medium.
- Strip "21st level" from epic destiny requirements.
- Normalise "shields" to "shield".
- Exclude background, theme, ritual, paragon path and epic destiny flavour text, and html-containing flavour text from search index.
- Include deity name in lookup index without "The".
- Include unique monster name in lookup index (instead of the combined name which never match anything).
- Add "reptile" keyword to Draconians race.
- Fix ability score bonus of race64 (Llewyrr Elf) and race68 (Kapak Draconian).
- Fix price of item1895 (Mrtok, Ogre Chief).
- Fix monster3717 (Blistered Soul) missing creature size and type.
- Fix item1864 (Mirror of Deception) missing power frequency, and remove line break in body text.
- Fix blank lookup matching glossary670 (Magic Item).
- Normalise trap1019 (Rubble Topple) type and role.
- Normalise epicdestiny698 (Unyielding Sentinel) prerequisite.
- Shorten benefits of background30 (Scorned Noble), background570 (Urban Shaman), and eight other Dragon 371 backgrounds.
- Shorten power Type "Encounter" to "Enc." and "Immediate" to "Imm.".
- Shorten "Draconomicon" to "Draco".
- Default sort Hybrid class last.
- Default sort powers by class, level, type (new), then name.
- [Regression] Fix some artifact's "published in" text.
  • Loading branch information
Sheep-y committed Dec 15, 2017
2 parents 872b9a4 + f071e8a commit eb33779
Show file tree
Hide file tree
Showing 74 changed files with 5,988 additions and 1,457 deletions.
43 changes: 24 additions & 19 deletions README.md
@@ -1,74 +1,79 @@
# 4e Compendium Downloader #

## Version 3.5.3 ##
## Version 3.6 ##

[![3 min. intro video](https://raw.githubusercontent.com/Sheep-y/trpg-dnd-4e-db/development/res/img/Frontpage.jpg)](https://youtu.be/aNDze9Ok5fE)
[![3 min. intro video](https://raw.githubusercontent.com/Sheep-y/trpg-dnd-4e-db/master/res/img/Frontpage.jpg)](https://youtu.be/aNDze9Ok5fE)

This app can fetches, stores, and exports D&D 4th edition resources from [D&D Compendium](http://www.wizards.com/dndinsider/compendium/database.aspx). <br/>
This program can fetches, stores, and exports D&D 4th edition resources from [D&D Compendium](http://www.wizards.com/dndinsider/compendium/database.aspx). <br/>
Exported entries can be browsed and searched offline with exact phase search, wildcast, exclusion, join ("OR"), and more.

本應用能從四代龍與地下城的[官方資料庫](http://www.wizards.com/dndinsider/compendium/database.aspx)提取、儲存、及匯出資料,以便進行離線的威力搜尋。<br/>
下載器的說明只限英語。匯出資料後會有中文操作說明。
下載器的說明只限英語。匯出資料後的搜尋介面可以在說明頁中切換成中文。
注意 3.x 為最後一代有中文介面的數據庫。日後的版本只支援英文。

This is a fan project and does not come with copyrighted data.

## How To Download Data ##

1. You need an active [Dungeons & Dragons Insider subscription](http://ddi.wizards.com/) to fetch data.
1. This may be impossible since registration is closed. See [in-app help](http://htmlpreview.github.io/?https://github.com/Sheep-y/trpg-dnd-4e-db/blob/development/res/downloader_about.html#faq) for links.
1. A new subscription can be purchased from [digitalriver](http://gc.digitalriver.com/store/dndi/html/pbPage.wizards).
2. [Download](http://www.java.com/) and install Java (version 8 or above).
3. [Download](https://github.com/Sheep-y/trpg-dnd-4e-db/releases/) the downloader exe (Windows) or downloader jar (Linux/Mac).
4. Open a folder for the downloader, put it in, and run it.
1. Jar version: If double clicking the jar file does not work, open console and run "java -jar 4e_compendium_downloader.jar". This should launch the app.
1. Jar version: If double clicking the jar file does not work, open console and run "java -jar 4e_compendium_downloader.jar". This should launch the program.
5. In the downloader, fill in DDI username and password, then click "Download".
1. Download can be stopped and resumed any time.
2. See [in-app help](http://htmlpreview.github.io/?https://github.com/Sheep-y/trpg-dnd-4e-db/blob/development/res/downloader_about.html) for steps and troubleshoots.
2. See [in-program help](http://htmlpreview.github.io/?https://github.com/Sheep-y/trpg-dnd-4e-db/blob/master/res/downloader_about.html) for steps and troubleshoots.
6. Once all data is downloaded, you can export the data to an HTML file, which can be opened in browsers.
7. There is a [video guide](https://youtu.be/aNDze9Ok5fE) for the above steps.

### Compatibility ###

The downloader is tested with Sun Java version 8 update 121.
The downloader is tested with Sun Java 8 and 9.
If you use [OpenJDK](http://openjdk.java.net/), you also need [OpenJFX](http://openjdk.java.net/projects/openjfx/). <br/>
The viewer is tested on Chrome 56, Edge 14, Firefox 52, IE 11, Android Chrome 56, and Android Firefox 52. <br/>
The viewer is tested on Chrome 62, Edge 15, Firefox 57, IE 11, Android Chrome 62, and Android Firefox 57. <br/>
Mac and iOS cannot be supported. Walled garden is walled.

## Different from Official Compendium ##
## Differences from Official Compendium ##

This downloader will create new data columns for easier data filter:

* Race: Origin (and keywords).
* Background: Benefit (Replace Associated Skills).
* Theme: Prerequisite.
* Theme: Prerequisites.
* Power: Type (Frequency + Type) and Keyword.
* Feat: Prerequisite.
* Item: Split into 4 categories (Item, Weapon, Implement, and Armor). Add Type column.
* Terrain: Group and Level. Combine with Trap for better mobile layout.
* Companion: Size and Type.
* Monster: Size and Type.
* Terrain: Group and Level. Combine with Trap to control category count.

Many existing columns are also enhanced or corrected.
Many existing columns are also enhanced or corrected, such as multi-source classes.

### Fixed errors ###

Let's face it, the official compendium has errors.
This downloader fixes over 1300 entries during export:
This downloader fixes over 1560 entries during export:

* 187 entries moved to correct category. (Superior implements, Lair items, Assassin's poisons, Consumable wondrous)
* 150 items missing power frequency. (Dantrag's Bracers, many energy weapons, dragon orbs, light sources etc.)
* 88 empty entries or non-resources, removed. (male, female, fang titan drake, Fastpaw background, etc.)
* 67 entries without "published" record. (Granny's Grief, Dreamheart, Deck of Many Things etc.)
* 60 entries moved to correct category. (Superior implements, Assassin's poisons, and Consumable wondrous)
* 35 entries that wrongly says "basic melee attack". (Bane's Tactics, Feral Armor, Dancing Weapon, Kobold Piker etc.)
* 24 entries with missing/wrong content. (Kord's Relentlessness, Rings of the Akarot, Hybrid Vampire Surge etc.)
* 23 entries with formatting issues. (Mirror of Deception, Silver Hands of Power, etc.)
* 20 entries with missing/wrong content. (Kord's Relentlessness, Rings of the Akarot, Hybrid Vampire Surge etc.)
* 10 typos. (Cambion Stalwart "bit points", Primal Grove "grp", Rubble Topple "Singe", Essential hybrid sourcebook etc.)
* 80 entries with other inconsistent content. (Winged Armor type, Xenda-Dran Array tier, Racial power without type etc.)
* 2 new entries (Artifact and Item Set.)
* 204 entries with other inconsistent content. (Winged Armor type, Xenda-Dran Array tier, Racial power without type etc.)
* 94 entries with missing listing data not caused by above issues. (Subrace abilities, Heroic feat tier etc.)
* 733 entries with wrong/inconsistent listing data not caused by above issues. (Multi-source classes, theme powers etc.)
* 724 entries with wrong/inconsistent listing data not caused by above issues. (Multi-source classes, theme powers etc.)
* In addition, over a hundred artifacts and item sets are manually classified.

If you find similar mistakes that aren't fixed, please [file an issue](https://github.com/Sheep-y/trpg-dnd-4e-db/issues/).

## Developer's Guide ##

This app has two parts: a downloader that fetch and export data, and a viewer that browse exported data.
This program has two parts: a downloader that fetch and export data, and a viewer that browse exported data.
The downloader will embed the viewer on build, so that there is only one deployable file.

* The [development branch](https://github.com/Sheep-y/trpg-dnd-4e-db/tree/development) is less vigorously tested but usually have more features and/or fixes.
Expand Down
5 changes: 5 additions & 0 deletions build.xml
Expand Up @@ -86,6 +86,7 @@
<fileset dir="org"/>
<fileset dir="db4e"/>
<fileset dir="sheepy"/>
<fileset dir="SevenZip"/>
<fileset dir="META-INF"/>
<fileset file="sqljet.build.properties"/>
</delete>
Expand All @@ -96,6 +97,10 @@
<!-- Compile viewer and copy to user home, will always rebuild -->
<target name="make_viewer" depends="-make-html" description="Compile the HTML viewer and moves it to default export location (user home)" >
<copy todir="${user.home}"><fileset file="res/4e_database.html"/></copy>
<copy todir="${user.home}/4e_database_files/res">
<fileset file="res/script.js"/>
<fileset file="res/style.css"/>
</copy>
</target>

<target name="-check-jar">
Expand Down

0 comments on commit eb33779

Please sign in to comment.