diff options
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 129 |
1 files changed, 44 insertions, 85 deletions
diff --git a/README.md b/README.md index 06480dee..7f354714 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,11 @@ # HSMusic -HSMusic, short for the *Homestuck Music Wiki*, is a revitalization and reimagining of [earlier][fandom] [projects][nsnd] archiving and celebrating the expansive history of Homestuck official and fan music. Roughly periodic releases of the website are released at [hsmusic.wiki][hsmusic]; all development occurs in this public Git repository, which can be accessed at [github.com][github]. +HSMusic, short for the *Homestuck Music Wiki*, is a revitalization and reimagining of [earlier][fandom] [projects][nsnd] archiving and celebrating the expansive history of Homestuck official and fan music. Roughly periodic releases of the website are released at [hsmusic.wiki][hsmusic]; all development occurs in a few different public Git repositories: + +- hsmusic-wiki ([GitHub][github-code], [Notabug][notabug-code], [cgit][cgit-code]): all the code used to run hsmusic on your own computer, and the canonical reference for all of the wiki's software behavior +- hsmusic-data ([GitHub][github-data], [Notabug][notabug-data], [cgit][cgit-data]): all the data representing the contents of the wiki; collaborative additions and improvements to wiki content all end up here +- hsmusic-media ([GitHub][github-media]): media files referenced by content in the data repository; includes all album assets, commentary images, additional files, etc +- hsmusic-lang ([GitHub][github-lang]): localization files for presenting the wiki's user interface in different languages ## Quick Start @@ -9,7 +14,7 @@ Install dependencies: - [Node.js](https://nodejs.org/en/) - we recommend using [nvm](https://github.com/nvm-sh/nvm) to install Node and keep easy track of any versions you've got installed; development is generally tested on latest but 20.x LTS should also work - [ImageMagick](https://imagemagick.org/) - check your package manager if it's available (e.g. apt or homebrew) or follow [installation info right from the official website](https://imagemagick.org/script/download.php) -Make a new empty folder for storing all your HSMusic repositories, then clone 'em with git: +Make a new empty folder for storing all your wiki repositories, then clone 'em with git: ``` $ cd /path/to/my/projects/ @@ -40,7 +45,7 @@ $ npm link # working out the details yourself), you can just move on. ``` -Go back to the main directory (containing all the repos) and make an empty folder for the first and subsequent builds: +Go back to the main directory (containing all the repos) and make a couple of empty folders that will be useful during builds: ``` $ cd .. @@ -53,131 +58,85 @@ code/ data/ media/ # Just do cd /path/to/my/projects/hsmusic (with whatever path you created # the main directory in) to get back. +$ mkdir cache $ mkdir out ``` -Then build the site: +**Anytime a command shows `hsmusic` in the following examples,** if your `npm link` command didn't work or you get a "command not found" error, you can just replace `hsmusic` with `node code`. -``` -# If you used npm link: -$ hsmusic --data-path data --media-path media --out-path out +The wiki uses thumbnails, but these aren't included in the media repository you downloaded. The wiki will automatically generate new thumbnails as you add them to the media repository (as part of each build), but the first time, you should just generate the thumbnails. -# If you didn't: -$ node code/src/upd8.js --data-path data --media-path media --out-path out +``` +$ hsmusic --data-path data --media-path media --cache-path cache --thumbs-only ``` -You should get a bunch of info eventually showing the site building! It may take a while (especially since HSMusic has a lot of data nowadays). +Provided you've got ImageMagick installed, this should go more or less error-free, although it may take a while (for the Homestuck Music Wiki, typically 40-80 minutes). It may fail to generate a few thumbnails, and will show an error message, if so. Just run the command again, and they should work the second time around. -If all goes according to plan and there aren't any errors, all the site HTML should have been written to the `out` directory. Use a simple HTTP server to view it in your browser: +Then build the site. There are two methods for this. **If you're publishing to the web** (or just want a complete, static build of the site for your own purposes), use `--static-build`, as below: ``` -$ cd site - -# choose your favorite HTTP server -$ npx http-server -p 8002 -$ python3 -m http.server 8002 -$ python2 -m SimpleHTTPServer 8002 +$ hsmusic --static-build --data-path data --media-path media --cache-path cache --out-path out ``` -If you don't have access to an HTTP server or lack device permissions to run one, you can also just view the generated HTML files in your browser and *most* features should still work. (Try `--append-index-html` in the `hsmusic`/`upd8.js` command to make generated links more direct.) This isn't an officially supported way to develop, so there might be bugs, but most of the site should still work. - -**If you encounter any errors along the way, or would like help getting the wiki working,** please feel welcomed to reach out through the [HSMusic Community Discord Server][discord]. We're a fairly active group there and are always happy to help! **This also applies if you don't have much experience with Git, GitHub, Node, or any of the necessary tooling, and want help getting used to them.** +The site's contents will generate in the specified `out` folder. For the Homestuck Music Wiki, this generally takes around 40-60 minutes. You can upload these to a web server if you'd like to publish the site online. Or run your HTTP server of choice (`npx http-server -p 8002`, `python3 -m http.server 8002`) to view the build locally. -## Project Structure +**If you're testing out your changes** (for example, before filing a pull request), use `--live-dev-server`, as below: -### General build process +``` +$ hsmusic --live-dev-server --data-path data --media-path media --cache-path cache +``` -When you run HSMusic to build the wiki, several processes happen in succession. Any errors along the way will be reported - we hope with human-readable feedback, but [pop by the Discord][discord] if you have any questions or need help understanding errors or parts of the code. +Once initial loading is complete (usually 8-16 seconds), the site will generate pages *as you open them in your web browser.* Open http://localhost:8002/ when hsmusic instructs you to. You have to restart the server to refresh its data and see any data changes you've saved; hold control and press C (^C) to cancel the build, then run the command again to restart the server. -1. Update thumbnails in the media repo so that any new images automatically get thumbnails. -2. Locate and read data files, processing them into relatively usable JS object-style formats. -3. Validate the data and show any errors caught during processing. -4. Create symlinks for static files and generate the basic directory structure for the site. -5. Generate and write HTML files (and any supporting files) containing all content. +### Help! It's not working -### Multiple repositories +**If you encounter any errors along the way, or would like help getting the wiki working,** please feel welcomed to reach out through the [HSMusic Community Discord Server][discord]. We're a fairly active group there and are always happy to help! **This also applies if you don't have much experience with Git, GitHub, Node, or any of the necessary tooling, and want help getting used to them.** -HSMusic works using a number of repositories in tandem: +### Building without writing `--data-path` (etc) every time -- [`hsmusic-wiki`][github-code] (colloquially "code"): The code repository, including all behavior required to process data and content from the other repositories and turn it into an actual website. This is probably the repo you're viewing right now. - - Code is written entirely in modern JavaScript, with the actual website a static combination of HTML and CSS (with inexhaustive JavaScript for certain features). - - More details about the code repository below. -- [`hsmusic-data`][github-data]: The data repository, comprising all the data which makes a given wiki what it *is*. The repository linked here is for the [Homestuck Music Wiki][hsmusic] itself, but it may be swapped out for other data repos to build other completely different wikis. - - This repo covers albums, tracks, artists, groups, and a variety of other things which make up the content of a music wiki. - - The data repo also contains all the metadata which makes one wiki unique from another: layout info, static pages (like "About & Credits"), whether or not certain site features are enabled (like "Flashes & Games" or UI for browsing groups), and so on. - - All data is written and accessed in the YAML format, and every file follows a specific structure described within this (code) repository. See below and the `src/data` directory for details. -- [`hsmusic-media`][github-media]: The media repository, holding all album, track, and layout media used across the site in one place. Media and organization directly corresponds to entries in the data repository; generally the data and media repositories go together and are swapped out for another together. -- [`hsmusic-lang`][github-lang]: The language repository, holding up-to-date strings and other localization info for HSMusic. - - Strings and language info are stored in top-level YAML files within this repository. They're based off the `src/strings-default.yaml` file within the code repo (and don't need to provide translations for all strings to be used for site building). +(These specific instructions apply only for bash and zsh. If you're using another shell, e.g. on Windows, you can probably adapt the principles, but we don't have a ready-to-go script, yet. Sorry!) -The code repository as well as the data and media repositories are require for site building, with the language repo optionally provided to add localization support to the wiki build. +It can be mildly inconvenient to write (or remember to write, or copy-paste) the `--data-path data` option, and similar options, every time. hsmusic will also detect and use environment variables for these; if you specify them this way, you don't need to provide the corresponding command line options. -The path to each repo may be specified respectively by the `--data-path`, `--media-path`, and `--lang-path` arguments (when building the site or using e.g. data-related CLI tools). If you find it inconvenient to type or keep track of these values, you may alternatively set environment variables `HSMUSIC_DATA`, `HSMUSIC_MEDIA`, and `HSMUSIC_LANG` to provide the same values. One convenient layout for locally organizing the HSMusic repositories is shown below: +Suppose you've locally organized your wiki repositories as below: path/to/my/projects/ hsmusic/ + cache/ <empty directory, or cached generated files> code/ <clone of hsmusic-wiki> data/ <clone of hsmusic-data> media/ <clone of hsmusic-media> - out/ <empty directory> (will be overwritten) - env.sh + out/ <empty directory, or a static build> -The `env.sh` script shown above is a straightforward utility for loading those variables into the envronment, so you don't need to type path arguments every time: +Create an `env.sh` file inside the top-level `hsmusic` folder, containing `data`, `media`, etc. If your shell is **bash,** enter these contents: #!/bin/bash base="$(realpath "$(dirname ${BASH_SOURCE[0]})")" + export HSMUSIC_CACHE="$base/cache/" export HSMUSIC_DATA="$base/data/" export HSMUSIC_MEDIA="$base/media/" - # export HSMUSIC_LANG="$base/lang/" # uncomment if present export HSMUSIC_OUT="$base/out/" -Then use `source env.sh` when starting work from the CLI to get access to all the convenient environment variables. (This setup is written for Bash of course, but you can use the same idea to export env variables with your own shell's syntax.) - -### Code repository source structure +If your shell is **zsh,** enter these contents: -The source code for HSMusic is divided across a number of source files, loosely grouped together in a number of directories: - -- `src/` - - `data/` - - `things/`: Descriptors for individual types of data objects used across the wiki, notably including: - - `cacheable-object.js`: Backbone of how data objects (colloquially "things") store, share, and compute their properties - - `thing.js`: Common superclass for most data objects, with a bunch of utilities and common behavior - - `validators.js`: Convenient error-throwing utilities which help ensure properties set on data objects follow the right format - - `yaml.js`: Mappings from YAML documents (the format used in `hsmusic-data`) to things (actual data objects), and a full suite of utilities used to actually load that data from scratch - - `content/`: Functions which generate HTML content; these go from bite-sized, commonly reused utilities (like `linkTemplate`) all the way up to entire page definitions (like `generateArtistInfoPage`) - - `page/`: Definitions for page paths, mapping data objects to paths served over an HTTP server (or written to an output folder) and to the functions which actually generate those pages' content - - `write/`: Common utilities and output methods for controlling what hsmusic does to turn data and media into something you actually visit; these each define a variety of command-line arguments and are basically the interchangeable "second half" of upd8.js - - `live-dev-server.js`: Gets the site available for viewing as quickly as possible, generating and serving pages as they are requested from a local HTTP server; reacts live to code changes in the `content` directory - - `static-build.js`: Builds the entire site at once, writing all the output to one self-contained folder which can be uploaded to a static file server - - `repl.js`: Provides a convenient REPL to run filters and transformations on data objects right from the Node.js command line - - `static/`: Purely client-side files are kept here, e.g. site CSS, icon SVGs, and client-side JS - - `util/`: Common utilities which generally may be accessed from both Node.js or the client (web browser) - - `upd8.js`: Main entry point which controls and directs site generation from start to finish - - `gen-thumbs.js`: Standalone utility also called every time HSMusic is run (unless `--skip-thumbs` is provided) which keeps a persistent cache of media MD5s and (re)generates thumbnails for new or updated image files - - `listing-spec.js`: Descriptors for computations and HTML templates used for the Listings part of the site - - `url-spec.js`: Index of output paths where generated HTML ends up; also controls where `<a>`, `<img>`, etc tags link - - `file-size-preloader.js`: Simple utility for calculating size of files in media directory - - `strings-default.json`: Template for localization strings and index of default (English) strings used all across the site layout - -## Forking - -hsmusic is a relatively generic music wiki software, so you're more than encouraged to create a fork for your own archival or cataloguing purposes! You're encouraged to [drop us a link][feedback] if you do - we'd love to hear from you. - -## Pull Requests - -As mentioned, part of the focus of the hsmusic.wiki release, as well as most development since, has been to create a more modular and developer-friendly repository. So, on the curious chance anyone would like to contribute code to the repo, that's more possible now than it used to be! - -Still, for larger additions, we encourage you to [drop the main devs an email][feedback] or, better yet, [pop by the Discord][discord] before writing all the implementation code: besides code tips which might make your life a bit easier (questions are welcome), we also love to discuss feature designs and values while they're still being brainstormed! That way, nobody has to tell you there are fundamental ideas or implementation details that should be rebuilt from the ground up - the last thing we want is anyone putting hours into code that has to be replaced by another implementation before it ever ends up part of the wiki! + #!/usr/bin/env zsh + base=${0:a:h} + export HSMUSIC_CACHE="$base/cache/" + export HSMUSIC_DATA="$base/data/" + export HSMUSIC_MEDIA="$base/media/" + export HSMUSIC_OUT="$base/out/" -As ever, feedback is always welcome, and may be shared via the usual links. Thank you for checking the repository out! +Then use `source env.sh` when starting work from the CLI to get access to all the convenient environment variables. [discord]: https://hsmusic.wiki/discord/ [fandom]: https://homestuck-and-mspa-music.fandom.com/wiki/Homestuck_and_MSPA_Music_Wiki - [feedback]: https://hsmusic.wiki/feedback/ - [github]: https://github.com/hsmusic/hsmusic-wiki + [cgit-code]: https://nebula.ed1.club/git/hsmusic-wiki + [cgit-data]: https://nebula.ed1.club/git/hsmusic-data [github-code]: https://github.com/hsmusic/hsmusic-wiki [github-data]: https://github.com/hsmusic/hsmusic-data [github-lang]: https://github.com/hsmusic/hsmusic-lang [github-media]: https://github.com/hsmusic/hsmusic-media [hsmusic]: https://hsmusic.wiki + [notabug-code]: https://notabug.org/towerofnix/hsmusic-wiki + [notabug-data]: https://notabug.org/towerofnix/hsmusic-data [nsnd]: https://homestuck.net/music/references.html |