diff options
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 165 |
1 files changed, 137 insertions, 28 deletions
diff --git a/README.md b/README.md index 2a67c00d..22f6eb55 100644 --- a/README.md +++ b/README.md @@ -1,47 +1,156 @@ # HSMusic -HSMusic, short for the *Homestuck Music Wiki*, is a revitalization and reimagining of [earlier][fandom] [projects][nsnd] archiving and celebrating the expansive history of Homestuck official and fan music. Roughly periodic releases of the website are released at [hsmusic.wiki][hsmusic]; all development occurs in this public Git repository, which can be accessed at [notabug.org][notabug]. +HSMusic, short for the *Homestuck Music Wiki*, is a revitalization and reimagining of [earlier][fandom] [projects][nsnd] archiving and celebrating the expansive history of Homestuck official and fan music. Roughly periodic releases of the website are released at [hsmusic.wiki][hsmusic]; all development occurs in a few different public Git repositories: -## Project Structure +- hsmusic-wiki ([GitHub][github-code], [Notabug][notabug-code], [cgit][cgit-code]): all the code used to run hsmusic on your own computer, and the canonical reference for all of the wiki's software behavior +- hsmusic-data ([GitHub][github-data], [Notabug][notabug-data], [cgit][cgit-data]): all the data representing the contents of the wiki; collaborative additions and improvements to wiki content all end up here +- hsmusic-media ([GitHub][github-media]): media files referenced by content in the data repository; includes all album assets, commentary images, additional files, etc +- hsmusic-lang ([GitHub][github-lang]): localization files for presenting the wiki's user interface in different languages -**Disclaimer:** most of the code here *sucks*. It's been shambled together over the course of over a year, and while we're fairly confident it's all at minimum functional, we can't guarantee the same about its understandability! Still, for the official release of [hsmusic.wiki][hsmusic], we've done our best to put together a codebase which is *somewhat* navigable. The description below summarizes it: +## Quick Start -* `src/upd8.js`: "Build" code for the site. Everything specific to generating the structure and HTML content of the website is conatined in this file. As expected, it's pretty massive, and is currently undergoing some much-belated restructuring. -* `src/static`: Static code and supporting files. Everything here is wholly client-side and referenced by the generated HTML files. -* `src/common`: Code which is depended upon by both client- and server-side code. For the most part, this is constants such as directory paths, though there are a few handy algorithms here too. -* In the not quite so far past, we used to have `data` and `media` folders too. Today, for portability and convenience in project structure, those are saved in separate repositories, and you can pass hsmusic paths to them through the `--data-path` and `--media-path` options, or the `HSMUSIC_DATA` and `HSMUSIC_MEDIA` environment variables. - * Data directory: The majority of data files belonging to the wiki are here. If you were to, say, create a fork of hsmusic for some other music archival project, you'd want to change the files here. Data files are all a custom text format designed to be easy to edit, process, and maintain; they should be self-descriptive. - * Media directory: Images and other static files referenced by generated and static content across the site. Many of the files here are cover art, and their names match the automatically generated "kebab case" identifiers for tracks and albums (or a manually overridden one). -* Same for the output root: previously it was in a `site` folder; today, use `--out-path` or `HSMUSIC_OUT`! +Install dependencies: -The upd8 code process was politely introduced by 2019!us back when we were beginning the site, and it's essentially the same structure followed today. In summary: +- [Node.js](https://nodejs.org/en/) - we recommend using [nvm](https://github.com/nvm-sh/nvm) to install Node and keep easy track of any versions you've got installed; during development we generally test on 22.x LTS +- [ImageMagick](https://imagemagick.org/) - check your package manager if it's available (e.g. apt or homebrew) or follow [installation info right from the official website](https://imagemagick.org/script/download.php) -1. Locate and read data files, processing them into relatively usable JS object-style formats. (The formats themselves are hard-coded and somewhat arbitrary, and are often extended when more or different data is useful.) -2. Validate the data and show any errors that might've been caught during processing. (These aren't exhaustive test cases; they're designed to catch a majority of common errors and typos.) -3. Create symlinks for static files and generate the basic directory structure for the site. -4. Generate and write HTML files containing all content. (Rather than use external templates and a complex build system, we just use template strings in combination with [a whitespace utility][fixws] and some handy tricks for manipulating strings and JS.) +Make a new empty folder for storing all your wiki repositories, then clone 'em with git: -The majority of the code volume is generated HTML content and supporting utility functions; while we've attempted to keep the update file more or less organized, the most reliable way to navigate is to just ctrl-F for the function definitions of whatever you intend to work on. Code order isn't super strict since everything is handled by separate function calls (which all branch off of the "main" function at the end of the file). +``` +$ cd /path/to/my/projects/ +$ mkdir hsmusic +$ cd hsmusic +$ git clone https://github.com/hsmusic/hsmusic-wiki code +Cloning into 'code'... +$ git clone https://github.com/hsmusic/hsmusic-data data +Cloning into 'data'... +$ git clone https://github.com/hsmusic/hsmusic-media media +Cloning into 'media'... +``` -In the past, data, HTML, and media files were all interspersed with each other. Yea, even the generated HTML files were included as part of the repository; their diffs, part of every commit. Those were dark times indeed. +Install NPM dependencies (packages) used by HSMusic: -## Forking +``` +$ cd code +$ npm install +added 413 packages, and audited 612 packages in 10s +``` -hsmusic is a relatively generic music wiki software, so you're more than encouraged to create a fork for your own archival or cataloguing purposes! You're encouraged to [drop us a link][feedback] if you do - we'd love to hear from you. +Optionally, use `npm link` to make `hsmusic` available from the command line anywhere on your device: -Still, at present moment, a fair bit of the wiki design is baked into the update code itself - any configuration (such as getting rid of the "flashes & games") section will have you digging into the code yourself. In the future, we'd love to make the wiki software more customizable from a forking perspective, but we haven't gotten to it yet. Let us know if this is something you're interested in - we'd love to chat about what additions or changes would be useful in making a more versatile generic music wiki software! +``` +$ npm link +# This doesn't work reliably on every device. If it shows +# an error about permissions (and you aren't interested in +# working out the details yourself), you can just move on. +``` -## Pull Requests +Go back to the main directory (containing all the repos) and make a couple of empty folders that will be useful during builds: -As mentioned, part of the focus of the hsmusic.wiki release was to create a more modular and develop-able repository. So, on the curious chance anyone would like to contribute code to the repo, such is certainly capable now! +``` +$ cd .. -Still, for larger additions, I'd encourage you to throw an email or contact ([links here][feedback]) before writing all the implementation code: besides code tips which might make your life a bit easier (questions are welcome), I'd also love to discuss feature designs and values while they're still being brainstormed! That way, I don't need to tell you there are fundamental ideas or code details I'd want rebuilt - the last thing I want is anyone putting hours into code which could have been avoided being poured down the drain! +$ pwd +/path/to/my/projects/hsmusic +$ ls +code/ data/ media/ +# If you don't see the above info, you've moved to the wrong directory. +# Just do cd /path/to/my/projects/hsmusic (with whatever path you created +# the main directory in) to get back. -As ever, feedback is always welcome, and may be shared via the usual links. Thank you for checking the repository out! +$ mkdir cache +$ mkdir out +``` +**Anytime a command shows `hsmusic` in the following examples,** if your `npm link` command didn't work or you get a "command not found" error, you can just replace `hsmusic` with `node code`. + +The wiki uses thumbnails, but these aren't included in the media repository you downloaded. The wiki will automatically generate new thumbnails as you add them to the media repository (as part of each build), but the first time, you should just generate the thumbnails. + +``` +$ hsmusic --data-path data --media-path media --cache-path cache --thumbs-only +``` + +Provided you've got ImageMagick installed, this should go more or less error-free, although it may take a while (for the Homestuck Music Wiki, typically 40-80 minutes). It may fail to generate a few thumbnails, and will show an error message, if so. Just run the command again, and they should work the second time around. + +Then build the site. There are two methods for this. **If you're publishing to the web** (or just want a complete, static build of the site for your own purposes), use `--static-build`, as below: + +``` +$ hsmusic --static-build --data-path data --media-path media --cache-path cache --out-path out +``` + +The site's contents will generate in the specified `out` folder. For the Homestuck Music Wiki, this generally takes around 40-60 minutes. You can upload these to a web server if you'd like to publish the site online. Or run your HTTP server of choice (`npx http-server -p 8002`, `python3 -m http.server 8002`) to view the build locally. + +**If you're testing out your changes** (for example, before filing a pull request), use `--live-dev-server`, as below: + +``` +$ hsmusic --live-dev-server --data-path data --media-path media --cache-path cache +``` + +Once initial loading is complete (usually 8-16 seconds), the site will generate pages *as you open them in your web browser.* Open http://localhost:8002/ when hsmusic instructs you to. You have to restart the server to refresh its data and see any data changes you've saved; hold control and press C (^C) to cancel the build, then run the command again to restart the server. + +### Help! It's not working + +**If you encounter any errors along the way, or would like help getting the wiki working,** please feel welcomed to reach out through the [HSMusic Community Discord Server][discord]. We're a fairly active group there and are always happy to help! **This also applies if you don't have much experience with Git, GitHub, Node, or any of the necessary tooling, and want help getting used to them.** + +### Building without writing `--data-path` (etc) every time + +(These specific instructions apply only for bash and zsh. If you're using another shell, e.g. on Windows, you can probably adapt the principles, but we don't have a ready-to-go script, yet. Sorry!) + +It can be mildly inconvenient to write (or remember to write, or copy-paste) the `--data-path data` option, and similar options, every time. hsmusic will also detect and use environment variables for these; if you specify them this way, you don't need to provide the corresponding command line options. + +Suppose you've locally organized your wiki repositories as below: + + path/to/my/projects/ + hsmusic/ + cache/ <empty directory, or cached generated files> + code/ <clone of hsmusic-wiki> + data/ <clone of hsmusic-data> + media/ <clone of hsmusic-media> + out/ <empty directory, or a static build> + +Create an `env.sh` file inside the top-level `hsmusic` folder, containing `data`, `media`, etc. If your shell is **bash,** enter these contents: + + #!/bin/bash + base="$(realpath "$(dirname ${BASH_SOURCE[0]})")" + export HSMUSIC_CACHE="$base/cache/" + export HSMUSIC_DATA="$base/data/" + export HSMUSIC_MEDIA="$base/media/" + export HSMUSIC_OUT="$base/out/" + +If your shell is **zsh,** enter these contents: + + #!/usr/bin/env zsh + base=${0:a:h} + export HSMUSIC_CACHE="$base/cache/" + export HSMUSIC_DATA="$base/data/" + export HSMUSIC_MEDIA="$base/media/" + export HSMUSIC_OUT="$base/out/" + +Then use `source env.sh` when starting work from the CLI to get access to all the convenient environment variables. + +## Notes for developers + +### Which platform features are OK for hsmusic-wiki!? + +Here's how to decide when new programming language features are ready for use in hsmusic-wiki: + +- **Build-code JS** (anything *not* in the browser): Whatever is supported in the latest LTS release of node.js. We hung out on 20.x for a needlessly long time; now we're calling that we can update to the latest LTS at any point. + +- **Client JS:** This one's mucky, but target stuff available across latest browsers since three years ago (to the month). It's generally possible for individual client modules to fail and even for graceful fallbacks where we notice, so it's not the end of the world for stuff to slip through a little early. + +- **Client HTML and CSS:** Firmer focus on only using what was available across browsers three years ago (to the month). Utterly ornamental CSS can use newer features, but only if it's flat-out necessary and does not impact the usability of the site at all (i.e. if the CSS fails then it's just as though that ornament weren't added). + +We haven't noted whether caniuse and MDN are based on different data sets or not, but *out of convenience,* we usually use MDN now that it shows this "Baseline" box for a lot of features, with month-year compat info up front. We're aware that these resources are never perfect, so pay closer attention before committing to huge new JS APIs, etc. + + [discord]: https://hsmusic.wiki/discord/ [fandom]: https://homestuck-and-mspa-music.fandom.com/wiki/Homestuck_and_MSPA_Music_Wiki - [nsnd]: https://homestuck.net/music/references.html + [cgit-code]: https://nebula.ed1.club/git/hsmusic-wiki + [cgit-data]: https://nebula.ed1.club/git/hsmusic-data + [github-code]: https://github.com/hsmusic/hsmusic-wiki + [github-data]: https://github.com/hsmusic/hsmusic-data + [github-lang]: https://github.com/hsmusic/hsmusic-lang + [github-media]: https://github.com/hsmusic/hsmusic-media [hsmusic]: https://hsmusic.wiki - [notabug]: https://notabug.org/hsmusic/hsmusic - [fixws]: https://www.npmjs.com/package/fix-whitespace - [feedback]: https://hsmusic.wiki/feedback/ + [notabug-code]: https://notabug.org/towerofnix/hsmusic-wiki + [notabug-data]: https://notabug.org/towerofnix/hsmusic-data + [nsnd]: https://homestuck.net/music/references.html |