XPiratez Research Tree Explorer (aka xresearch)

xresearch is a web-based application meant to allow exploration of the XPiratez research tree data. The contents of this repository’s /docs directory are the static website available at https://olsonjeffery.github.io/xresearch.

This web application is released into the public domain with no license, copyright or warranty claims made. Dependent libraries carry their own licenses as part of their source (or implied by the license available at their repository/website-of-record).

For better or worse, this app was developed and tested in Chrome, so use that if you want it to run w/ relatively little hassle.

This application was conceived-of and developed-by Jeffery Olson for the sake of exploring XPiratez’s in-game research-tree data (which is already explorable via an in-game “tech viewer”) and as an opportunity to develop with tools like React, cytoscape.js, etc.

Dependencies

• XPiratez, of course, without whom we wouldn’t have the glorious data to explore
  • xresearch relies upon a copy of the Piratez.rul and Piratez_lang.rul files, provided as a part of the XPiratez mod distro. They may have been modified to address YAML parsing issues, only.
• node.js for the ruleset parser and dev server
• js-yaml for YAML parsing (the format for the ‘.rul’ file representation)
• simple-grid.css for a barebones CSS grid
• lodash is great; everyone should use it
• cytoscape.js a wonderful and performant graph viz library (funded with public money, no less!)
• redux for view state management
• react as a view engine and structural pattern
• font-awesome for icons

HOWTO: Run development server locally

1. have recent node.js + npm installed (also webpack should be installed and available from the CLI)
2. clone this repository to your local machine
3. from the CLI, enter the repo directory and run ‘npm install’
4. ‘npm run dev’ to bring up the development server
5. Once you see “data is parsed” displayed in the output, you can navigate to ‘http://localhost:3000’ in Chrome to see the app
6. You should also run the ‘webpack’ command within the repo (which includes a ‘webpack.config.js’) in order to update the ‘/docs/bundle.js’ file as you make changes

HOWTO: pdate the Piratez.rul/Piratez_lang.rul files

1. Get the latest Xpiratez distro and unzip/unrar it somewhere
2. Pull out the ‘Piratez.rul’ and ‘Piratez_lang.rul’ files from ‘Dioxine_XPiratez/user/mods/Piratez/Ruleset/’ and overwrite the same files in the root of this repository
3. Run ‘npm run genstatic’ to attempt to generate the static HTML output
4. Errors will likely result, pointing to YAML parsing issues in the ‘.rul’ files brought over previously
5. Resolve YAML parsing issues until ‘npm run genstatic’ completes successfully
6. Your ruleset files are now updated

HOWTO: Regenerate static HTML (only needed if index_template.html or parser logic changes, resulting in JSON data differences)

1. Run ‘npm run genstatic’

CHANGELOG

• PENDING [0/17]

  [ ] FEATURE: add requiredToPurchase in rulesetParser/nodeList

  [ ] FEATURE: aggregate/list views as per the xp wiki

  • probably should drive it off of react-table

  [ ] a top-level /list route with all supported with implemented lists

  [ ] a sidebar list with implemented lists

  • [ ] crafts::

  [ ] weapons

  [ ] armor

  [ ] units: gals etc

  [ ] units: enemies

  [ ] research topics & base time

  [ ] FEATURE: router support in URL

  • [ ] all routes represent redux actions

  [ ] FEATURE: prefix clickable node list entries w/ colored icon for their ruleset kind

  • [ ] items: briefcase
  • [ ] facilities: building/building-o
  • [ ] ideas: lightblub-o
  • [ ] craft: fighter-jet
  • [ ] they shouldn’t change color on highlight

  [ ] FEATURE: item count in sidebar tables

  [ ] REFACTOR: pay down technical debt and improve composition in NodeListComponents

  • [ ] exported components themselves should move into their own modules
  • [ ] UX study of different nodelist component types and how the build* utilities are used
    • [ ] create a mockup of UX components
    • [ ] all of the build* helper funcs should be in a util module at the top level of a nodelist directory that contains the other modueles
    • [ ] possibly move away from build* methods and use actual components for different row types within the top-level nodelist tables
  • [ ] a look at composition vs data config (ie how SidebarNodeListCompoent is overloaded via props for use by SearchResultsListComponent vs GraphNodeTopicListComponent and how that pattern isn’t re-used anywhere else)

  [ ] FEATURE: parse this document and build JSON changelog/todo list and show in PageSplashComponent

  [ ] REFACTOR: audit/fix getOneFree relationships in parser, node details and graphcomponent

  • [ ] “giveOneFree” should be “gottenForFreeFrom”
  • [ ] getOneFree should move to the righthand side, relationship-wise, if it isn’t
  • [ ] fix the edge mapping for getOneFree entries (or just map off of gottenForFreeFrom)
  • [ ] fix style to have arrows always pointing in the right direction

  [ ] FEATURE: add graph legend

  [ ] FEATURE: “recent nodes” list via localStorage

  • [ ] add redux store listener to catch all nodeSelection actions
  • [ ] maintain running list of ten-most-recent nodes visited
  • [ ] maintain master list of all nodes visited & total visits (maybe build a dykstra map off of that for significant node display?)

  [ ] FEATURE: “significant nodes” summaries in show-all/initial display

  • [ ] when multiple nodes are displayed or during show-all-topics, show a special summary instead
  • [ ] top-10 most-linked
  • [ ] top-10 most needed to manufacturing

  [ ] FEATURE: effort-value-sell ratio

  • [ ] top-10 effort-value-sell ratio, possible algo:
    • [ ] compute an “effort value” for each item (sum of child-component “effort value” + buildTime*(cost or 1, whichever is greater)
    • [ ] compute EVS ratio by doing costSell/ev

  [ ] FEATURE: view raw JSON payload for a given node details

  • [ ] a json tree visualization library to display within a div…

  [ ] FEATURE: allow “multipled selected nodes” in graph display

  • [ ] use cases:
    • [ ] “initial search topics”: Call A Meeting, Visit Nearby Town, Inspect Machinery

  [ ] FEATURE: UX & hover polish

  • [ ] hovering on node list entry highlights matching edge in cytoscape
    • [ ] add edge id (FOO->BAR) to link entries
    • [ ] on hover-enter, find matching entity via __cy.$() and change its style
    • [ ] on hover-exit, find again and change style back
  • [ ] hovering over node list <td> triggers :hover that sets color to match its header coloring
  • [ ] hovering on node list title highlights all matching edge-styles in ctyoscape
    • [ ] make all edge styles match existing edgeNames and add styles for the inverse of existing edgeNames (+dependedUponBy, unlockedBy, giveOneFree)
    • [ ] add `edgeHighlight: string` to state
    • [ ] add SET_EDGE_HIGHLIGHT action type w/ args of “edgeName”
    • [ ] GraphComponent listens for edgeHighlight changes and sets the line-bolding in the approp style and restyles the graph
  • [ ] hovering on a node in the graph component highlights the entry in its nodelist

  [ ] BUG: only remove node on graph filtering changes if all of its dependency links to the selected node are removed (currently hides a node if any are unchecked)

  [ ] BUG: deal with SYNTH_CORPSE_ETC entries and their missing lang info

  • [ ] items with both ‘type’ and ‘name’ entries (all corpses)
  • [ ] the ‘type’ STR_ values all point to stuff not in the lang
  • [ ] maybe just remove them

• VERSION 0.1.1 [9/10]

  [ ] FEATURE: show manufacturing outputs, not just manufacturing inputs

  • more useful for rob/slavery

  [X] FEATURE: include/show buildCostItems data in facilities

  • [X] add buildCostItems in NodeList / Details
  • [X] test/impl in rulesetParser
  • [X] add requiredToConstruct in NodeList / Details
  • [X] build out graph edges for requiredtoConstruct

  [X] FEATURE: show “provides labs” in node trivia is present on facilities

  [X] FEATURE: add craft data

  • [X] add craft data in payload

  [X] FEATURE: integrate requiresBaseFunc/provideBaseFunc

  • [X] map providesBaseFunc in facilities entries to baseFuncKey->facilities STR_ keys
  • [X] show “provides” entries in facilities that provide them
  • [X] show requires entries + matching facilities in entries that require them
  • requires shows up in manufacture
  • provides shows up in facilities

  [X] FEATURE: when a graphFilteringCategory is deselected, make its displayed elements read-only and collapse the node list

  [X] FEATURE: build time display to factor in the number of runts assigned

  • [X] modify “manufacture time” entry in node trivia list to include num input and link to component state
  • [X] add unit test for parseBuildTime to include different run counts

  [X] FEATURE: collapsable node list components

  [X] FEATURE: UX overhaul

  • [X] create some mockups
    • [X] existing UX layout w/ left/right asides and center graph
    • [X] half/half layout with graph on right and left/right navs on other side side-by-side
  • [X] add intro page instead of show all, based on ux mockup
  • [X] no more “show all topics”
  • use org-mode-parser npm package from within ./ruleset-parser.js

  • [X] REFACTOR: get rid of all the uses of export default
• VERSION 0.1.0 [8/8]

  [X] FEATURE: handling research topics from the vanilla (e.g. STR_ALIEN_TERROR)

  • [X] make parse not complain w/ loading of unmodified YAML
  • [X] write script to pull in all vanilla data into repo alongside Piratez.rul, etc
    • vanilla stuff is at standard\xcom1
  • [X] PARSER: Integrate “vanilla language files” in common/en-US.yml , standard/xcom1/Language , etc for more str keys
    • [X] API to merge n ruleset branches together (eg research, manufacture, etc)

  [X] REFACTOR: remove “sidebar mode” logic in redux state

  [X] FEATURE: use COLOR_ORANGE for manufacturing edges

  [X] introduce unit testing framework

  • [X] server-side testing
    • [X] parser can handle ruleset
    • [X] parser loads manufacturables
    • [X] parser loads items
    • [X] facilities
    • [X] all relationships
  • [X] browser testing
    • [X] lunr module
    • [X] set up /test handler in express and add test_runner.html
    • [X] xrdataqueries

  [X] REFACTOR: move lunr into own module from NodeListComponent.js

  [X] FEATURE: Mobile UX

  • [X] use viewport detection stuff to fork between two different layout arrangements
    • [X] mobile: three rows, top row is just search results, 2nd row is graph, 3rd row is left, then right details
    • [X] desktop: existing setup
  • [X] graph has 100% width and height of viewport - 50px or something..

  [X] FEATURE: implement scrollbar-overflow for large viewport

  [X] REFACTOR: switch CSS grid/styling to bootstrap

  • [X] put drop shadow on tables
  • [X] retire node summary; put cost, points, etc into a new left-nav summary box
  • [X] graph work
    • [X] put border on graph component w/ drop shadow
    • [X] make background a darker gray
    • [X] integrate slate theme colors into graph component for node/edge colors
  • [X] rework node list display to use tables, instead of ul, and share more formatting code
    • [X] preserve click behavior when aprop
    • [X] use slate theme colors instead of default red, green, blue & gray
  • [X] push selected node title into nav bar as a breadcrumb
  • [X] have show-all-topics as button to the right of selected node title
  • [X] make “xresearch vblah” be a link to root route
  • [X] replace xresearch eye with spinner when loading
  • [X] integrate customized “slate” theme from bs 3.3.7 (need .js files)
  • [X] use “input addon” to hold magnifying-glass symbol for search
  • try to keep existing sidebar & graph layouts

• VERSION 0.0.3 [13/13]

  [X] REFACTOR: all “multi-dispatch” instances in event handlers -> go into single actions

  [X] REFACTOR: split SharedSetup.js into Constants & StateMgmt modules

  [X] BUG: fix broken graph filtering checkboxes by restoring edgeName in node list

  [X] FEATURE: add facilities to graphNodes

  • [X] ‘facilities’ under the root
  • [X] a new node type
  • [X] show build cost in sumamry, and normalized build time display (days, hrs)
  • [X] handle facilities not overwritten by vanilla data

  [X] FEATURE: show STR_ id in node details

  [X] FEATURE: Add/Show manufactuables to graphNodes

  • [X] Add new nodelist setup/display for manufacture requirement entries
  • [X] show build time
  • [X] add ‘manufactureCost’

  [X] REFACTOR: remove ‘edgeName’ from NodeLinkListComponent; caller passes data

  • [X] rename NodeLinkListComponent->GraphNodeTopicListComponent

  [X] FEATURE: add/show items to graphNodes

  • [X] add ‘requires’ edges in graph display
  • [X] hide “Points” in header for those topics that lack it
  • [X] add a requiresBuy display for nodes that have them in Left node list or wahtever
  • [X] make all left/right bars optional based on existence and show requires + inverse
  • [X] refactoring for common research/item/manufacture building and display
  • [X] change ‘cost’ to ‘researchCost’ so we can have a ‘purchaseCost’ added
  • from Dioxine:
    • it ain’t as simple with items. First of all they need to have non-zero buy price to be even considered buyable
    • then they can have requiresBuy: or requires:
    • the latter overrules the former
    • same is true for craft except there is no “requiresBuy ”

  [X] BUG: outside-tree items (e.g. STR_ALIEN_RESEARCH) so be non-selectable

  • [X] in sidebar display
  • [X] in graph

  [X] REFACTOR rename researchData to graphNodes

  [X] REFACTOR: get rid of graphNodes as array; always key dict

  [X] FEATURE: add STR_ ids to lunr index

  [X] REFACTOR: remove xrData from store and factor all uses into a new XrDataQueries module

  • [X] audit/bottleneck all uses of state.xrData
  • [X] PARSER: add label as the str_ id if no lang entry is found; remove getLabelFromXrData

  [X] FEATURE: show ruleset version in page display

• VERSION 0.0.2 [5/5]
  • [X] BUG: “show all topics” is broken
  • [X] PARSE: include xresearch version in xrData
  • [X] REFACTOR: change ‘graph updating…’ to be a spinner
  • [X] FEATURE: add watches on ruleset-parser and index_template.html and reload in dev server
  • [X] FEATURE: add crash handler/error boundary at top-level to show error msg
• VERSION 0.0.1 [24/24]
  • [X] bug viewing STR_ALIEN_RESEARCH off of Raider Thug leads to crash
    • [X] not adding nodes for edges outside research tree; should work?
    • [X] solution: make topic ids w/ no entry in researchData be just text, not links (disabling node selection)
  • [X] add checkbox filtering to each node list category in both panes
    • [X] add detailFilteringCategories: { dependencies, dependedUponBy, unlocks, unlockedBy, getOneFree, giveOneFree } to state
      • [X] add SET_DETAIL_FILTERING_CATEGORY action type w/ args of { filteringCategory: ‘unlocks’, value: true }
      • [X] add reducer that switches on filteringCategory & action.type and sets the key in detailFilteringCategories with whatever is in `value`
    • [X] on graph draw for selected node display, remove nodes falling into any of the above detailFilteringCategories marked as false
    • [X] add checkboxes to NodeLinkListComponent (should probably take a memoized function doing an aprop dispatch of SET_DETAIL_FILTERING_CATEGORY); defaulting to true/checked
      • [X] add onFilteringCheckboxChange to NodeLinkListComponent that gets the toggle and dispatches SET_DETAIL_FILTERING_CATEGORY action w/ the filteringCategory marshalled in props and checkbox checked status as value
  • [X] fix bug with un-string-delim’d ‘event’ in GraphComponent:44
  • [X] add footer info w/ links to project repo, xpiratez, etc
  • [X] static HTML generation for running as a github site
  • [X] colorize Red/Green/Blue as it appears in node list descriptions
  • [X] fix bug with duplicate STR_DEEP_ONE entries under dependencies for STR_BUGEYE_VICTIM
  • [X] PARSER: move keysIndexMap into parser from sidebar or whatever
  • [X] update to 099H2 ruleset files
  • [X] expand UX to include left/right details pain
    • [X] validate data consistency of xresearch vs in-game tech viewer
    • [X] duplicate nodedetails & map left/right edge displayers:
      • [X] left
        • [X] dependencies
        • [X] unlocked-by
        • [X] get-for-free-from
      • [X] right
        • [X] depended-upon-by
        • [X] unlocks
        • [X] getOneFree (gives one for free, actually?)
    • [X] cytoscape polish
      • [X] show arrows
      • [X] sort out source/target designation
    • [X] the graph takes up a narrower viewport in the center
    • [X] PARSER: map inverses of data in .rul entries (get terminology from tech viewer in client) (should be O(2) ):
      • [X] depended-upon-by, unlocked-by, get-for-free-from
      • [X] PARSER: verify/add base cost and score points
    • [X] graph header
      • [X] a header atop the container div that shows the node of the selected node (or “All Topics” if an individual node isn’t selected)
      • [X] base research cost, points shown for selected node
      • [X] “show all topics X” button on r-hand side if node is selected
      • [X] a loading message displayed l-hand side of header that is triggered onNodeSelection and removed on layout.on(“stop”)
  • [X] add node details sidebar view (include <ul>’s with dependencies, unlocks, getOneFree, requires (non-interactive))
    • [X] display dependencies
    • [X] display unlocks
    • [X] display getOneFree
    • [X] display requires
  • [X] add Provider component in root of app and strip out manual passing of `store`
  • [X] error in displaying The Mutant Alliance
    • [X] some are legit nodes whose string lives in vanilla files (STR_WORKSHOP)
    • [X] some are non-tree events that aren’t research topics and probably also have their nodes in vanilla files (STR_ALIEN_TERROR)
  • [X] CYTOSCAPE: show sub-graph when NodeDetail is displayed for a given node
    • [X] trim the cytoscape graph display to contain the selected node (bolded/enlarged or whatever), along with its immediate neighbors and the edges that connect them
    • [X] try to use the “cose” layout
  • [X] CYTOSCAPE: Clicking on a graph node dispatches NODE_SELECTION action
  • [X] fix dark text coloring in page style
  • [X] write Piratez/Piratez_lang.rul files parse
  • [X] import parsed JSON into a static html display
  • [X] Basic, whole-tree display/visualization
  • [X] integrate react and refactor to component
  • [X] integrate webpack bundling
  • [X] node search + sidebar view
    • [X] utilizing the input on the top-right of the page, search the tree topic names for matches
    • [X] display them in a <ul> list on the sidebar
  • [X] when “searchText” is empty, return nothing; when > 20 results, return nothing
  • [X] Integrate redux for basic view-state management