* Org Flashcards
Spaced-repetition system for use with Emacs org-mode.

#+CAPTION: Review Demo
[[file:images/review.png]]

** Introduction
In the most abstract sense, this package deals with

1. Attaching timestamped review information to headlines
2. Querying all headings where reviews are due
3. Reviewing due *positions* of headings

As mentioned in step 3, a heading can have multiple *positions*,
e.g. to implement cloze-deletions where multiple items are reviewed
independently from each other.

In the reviewing step, display functions can be registered by card
type. This allows easy addition of user-defined card types without
having to think about storing and updating review data.

Review functions are called with point on the headline of the card
that should be reviewed and get passed a single argument,
the position to be reviewed.

They are expected to return either ~'quit~ to end the review or one of
~'again~, ~'hard~, ~'good~, ~'ease~, to rate the card.

See also:
- [[file:doc/alternative_applications.org][Alternative Applications]]
- [[file:doc/incremental_reading.org][Incremental Reading]] (todo)
** Design Goals / Choices
1. Use of =awk= for quickly finding cards due for review
2. Support for multiple *positions* in a card / heading
3. All relevant data kept in org files
   for easy version control
4. Review directly on the source org file
   for easy editing of cards during review

See [[file:doc/design_choices.org][Design Choices]] for more information.
** Prior Art
There are a few other packages for implementing a SRS based on org-mode.

Below, I've listed a the ones I've found so far that are actively
maintained and implement a lot of useful functionality.

- [[https://gitlab.com/phillord/org-drill/][phillord/org-drill]]
- [[https://github.com/abo-abo/pamparam][abo-abo/pamparam]]

Other (open source) flashcard / spaced repetition software:

- [[https://apps.ankiweb.net/][Anki]]
- [[https://mnemosyne-proj.org/][Mnemosyne Project]]

Thanks to the maintainers and all contributors for their work on these
packages!
** Performance
All user-facing commands (especially during review) should be as fast
as possible (<300ms).

Using the =awk= indexer, searching 2500 org files (~200k lines in
total) for due flashcards takes around ~500ms on my laptop (Thinkpad
L470, SSD).

Using the lisp indexer based on ~org-map-entries~,
searching a single 6500 line file with 333 flashcards takes ~1000ms,
indexing the same file with =awk= takes around ~50ms.
** Getting Started
Before using this package, ~org-fc-directories~
should be set to the directory to search for org files containing flashcards.

The file used to store the review history can be customized with
~org-fc-review-history-file~ and defaults to ~/path/to/config/org-fc-reviews.tsv~.

This package is not (yet) available on MELPA / ELPA,
to install it, clone the repository (e.g. to ~src/org-fc/~)
and follow the setup instructions.

*** Dependencies
- The =gawk= extension of =awk= for extracting review data from =.org= files
- =find= for finding all org files in the ~org-fc-directories~
- =xargs= for processing files in parallel
**** Linux / UNIX
=find= and =xargs= should be included in most distributions, =gawk=
can be installed from source or using your package manager of choice.

Examples:
- =pacman -S gawk= (Arch Linux)
- =apt-get install gawk= (Ubuntu, Debian, ...)
- =yum install gawk= (CentOS)

For more information, see [[https://www.gnu.org/software/gawk/manual/html_node/Installation.html][gawk manual - Installation]].
**** MacOS
On MacOS all dependencies can be installed using [[https://www.macports.org/][macports]] or [[https://brew.sh/][homebrew]].
=find= and =xargs= are part of the =findutils= package.

- =port install gawk findutils=
- =brew install gawk findutils=

You might have to adjust your =$PATH= to make sure Emacs can find all
relevant binaries.

#+begin_src bash
export PATH="/opt/local/libexec/gnubin:/opt/local/bin:$PATH"
#+end_src

For more information, refer to the documentation of macports /
homebrew.
*** Setup with [[https://github.com/jwiegley/use-package/][use-package]]
#+begin_src emacs-lisp
(use-package hydra)
(use-package org-fc
  :load-path "~/src/org-fc"
  :custom
  (org-fc-directories '("~/org"))
  :config
  (require 'org-fc-hydra))
#+end_src

Or, using [[https://github.com/raxod502/straight.el/][straight.el]]:

#+begin_src emacs-lisp
(use-package hydra)
(use-package org-fc
  :straight (org-fc :type git :host github :repo "l3kn/org-fc")
  :custom
  (org-fc-directories '("~/org/"))
  :config
  (require 'org-fc-hydra))
#+end_src

Note that in this case, you don't have to clone the repository.
*** Minimal Setup
Assuming [[https://github.com/abo-abo/hydra][abo-abo/hydra]] is already loaded.

#+begin_src emacs-lisp
(add-to-list 'load-path "~/src/org-fc/")
(setq org-fc-directories '("~/org"))
#+end_src
*** Default Hydra
[[file:org-fc-hydra.el]] defines a hydra for accessing commonly used
org-fc commands and for marking headlines as flashcards.

It can be loaded and bound to a hotkey like this:

#+begin_src emacs-lisp
  (require 'org-fc-hydra)
  (global-set-key (kbd "C-c f") 'org-fc-hydra/body)
#+end_src
*** Demo Mode
A file demonstrating all card types is included.
~M-x org-fc-demo~ starts a review of this file.

Note that the review data of the cards in this file *is not updated*.
** Known Issues
If you want to execute any other emacs command during review,
make sure to quit the review using the =q= key.

Otherwise the buffer is left in a narrowed state (this can be reset by
manually calling =M-x org-fc-review-quit=).
** Card Types
This package comes with a few predefined card types.
They are documented in [[file:doc/card_types.org][Card Types]].

[[file:demo.org][demo.org]] includes examples for each of these types.
** Marking Headlines as Cards
A *card* is an org-mode headline with a =:fc:= tag attached to it.
Each card can have multiple *positions* reviewed independently from
each other, e.g. one for each hole of a cloze card.

Review data (ease, interval in days, box, due date) is stored in a table
in a drawer inside the card.

#+begin_src org
  :REVIEW_DATA:
  | position | ease | box | interval | due                    |
  |----------+------+-----+----------+------------------------|
  |        2 | 2.65 |   6 |   107.13 |    2020-04-07T01:01:00 |
  |        1 | 2.65 |   6 |   128.19 |    2020-04-29T06:44:00 |
  |        0 | 2.95 |   6 |   131.57 |    2020-04-30T18:03:00 |
  :END:
#+end_src

Review results are appended to a csv file to avoid cluttering the org
files.

Each card needs at least two properties, an *unique* ~:ID:~ and a
~:FC_TYPE:~.  In addition to that, the date a card was created
(i.e. the headline was marked as a flashcard) is stored to allow
making statistics for how many cards were created in the last day /
week / month.

#+begin_src org
  :PROPERTIES:
  :ID:       4ffe66a7-7b5c-4811-bd3e-02b5c0862f55
  :FC_TYPE:  normal
  :FC_CREATED: 2019-10-11T14:08:32
  :END:
#+end_src

Card types (should) implement a ~org-fc-type-...-init~ command that
initializes these properties and sets up the review data drawer

All timestamps created and used by org-flashcards use ISO8601 format
with second precision and without a timezone (timezone UTC0).

This prevents flashcard due dates from showing up in the org-agenda
and allows filtering for due cards by string-comparing a timestamp
with one of the current time.
** (Un)suspending Cards
Cards can be suspended (excluded from review) by adding a =suspended=
tag, either by hand or using the ~org-fc-suspend-card~ command.

All cards in the current buffer can be suspended using the
~org-fc-suspend-buffer~ command.

The reason for using a per-headline tag instead of a file keyword is
that this way cards stay suspended when moved to another buffer.

Cards can be un-suspended using the ~org-fc-unsuspend-card~ and
~org-fc-unsuspend-buffer~ commands.

If the card being unsuspended was not due for review yet,
or was due less than 10% of its interval ago, its review data is not
reset. If it was due by more than that, its review data is reset to
the initial values.
** Spacing Algorithm
This package uses a modified version of the SM2 spacing algorithm,
based on the one used by Anki.

See also:
- [[file:doc/custom_spacing_algorithms.org][Custom Spacing Algorithms]] (todo)
- [[https://apps.ankiweb.net/docs/manual.html#what-algorithm][Anki Manual - Algorithm]]
- [[https://www.supermemo.com/en/archives1990-2015/english/ol/sm2][SuperMemo - SM2 Algorithm]]
** Review
Reviewing cards is done by opening the file the card is in,
using a special narrowing function to hide other headings
and drawers.

With ~(point)~ on the headline to be reviewed,
the setup function for this card type is called
(e.g. to hide the cloze holes of the card).

Then the flip function for the card type is called,
usually opening a *hydra* showing available hotkeys.

Once the card is flipped, another hydra for rating the card is shown.

A review session can be started using ~org-fc-review-all~
to review all cards that are due, or using ~org-fc-review-buffer~ to
review only cards in the current buffer.

The current review session can be ended / reset using
~org-fc-review-quit~.

Ideally, don't use any other hotkeys while in a review session.
This exits the review hydra without ending the current review session
making it necessary to do so manually (~org-fc-review-quit~).

~org-fc-tag-card~ can be used on a card heading or during review
to add one of the ~org-fc-card-tags~ to the card.

Tags can be added to this list using ~(add-to-list 'org-fc-card-tags
"my-tag")~.

See also:
- [[file:doc/review_internals.org][Review Internals]]
- [[file:doc/review_history.org][Review History]]
** Review Contexts
By default, two contexts are defined:

- all :: all cards in ~org-fc-directories~
- buffer :: all cards in the current buffer

New contexts can be defined by adding them to the alist
~org-fc-custom-contexts~.

Contexts have the form ~(:paths paths :filter filter)~.

- ~:paths~ (optional)
  either a list of paths, a single path
  or ~'buffer~ for the current buffer.
  Paths don't have to be included in the ~org-fc-directories~.
  Defaults to ~org-fc-directories~.
- ~:filter~ (optional), a card filter defaulting to a filter that
  matches all cards.

Filters can be combinations of the following expressions:

- ~(and ex1 ex2 ...)~
- ~(or ex1 ex2 ...)~
- ~(not ex)~
- ~(tag "tag")~
- ~(type card-type) or (type "card-type")~

*** Examples
All double cards with tag "math":
#+begin_src emacs-lisp
  (add-to-list 'org-fc-custom-contexts
    '(double-math-cards . (:filter (and (type double) (tag "math")))))
#+end_src

All cards in that don't have one of the tags "foo" and "bar":
#+begin_src emacs-lisp
  (add-to-list 'org-fc-custom-contexts
    '(no-foo-bar-cards . (:filter (not (or (tag "foo") (tag "bar"))))))
#+end_src

All cards in =~/combinatorics/= or =~/number_theory.org=:
#+begin_src emacs-lisp
  (add-to-list 'org-fc-custom-contexts
    '(math-cards . (:paths ("~/combinatorics/" "~/number_theory.org"))))
#+end_src

All cards in =~/combinatorics/= with tag "theorem":
#+begin_src emacs-lisp
  (add-to-list 'org-fc-custom-contexts
    '(combinatorics-theorems .
      (:paths "~/combinatorics/" :filter (tag "theorem"))))
#+end_src

All double cards in the current buffer:
#+begin_src emacs-lisp
  (add-to-list 'org-fc-custom-contexts
    '(current-double .
      (:paths buffer :filter (type double))))
#+end_src
*** Note
Because parsing of tags is done in AWK, tag filters don't work for
tags defined in the =#+FILETAGS:= of a =#+SETUP_FILE:=.
** Dashboard / Statistics
~org-fc-dashboard~ shows a buffer with statistics for review performance
and cards / card types.

[[file:images/stats.png]]

Only cards with a box >= ~org-fc-stats-review-min-box~ (default: 0)
are included in the review statistics.

Setting this to a higher value (e.g. 2) excludes the first few
"learning" reviews of a card.

See also:

- [[file:doc/review_history.org][Review History]]
- [[file:doc/advanced_statistics.org][Advanced Statistics]] (todo)
** Changelog
I hope the current card / log format is flexible enough to accommodate
upcoming changes.

In case a update to the org sources is needed, I'll add a changelog
entry with updating instructions.
*** [2020-05-01 Fri]
**** Internal
The AWK scripts now generate S-expressions instead of CSV tables, this
way ~read~ can be used to parse the data instead of relying on a set
of custom parsing functions.

This also allows passing more complex data structures from AWK to
org-fc.
*** [2020-04-29 Wed]
Implemented a new version of the spacing algorithm (SM2) that's used
by org-fc.

The only difference is in how the next interval for cards rated as
"hard" is calculate.

The initial version (~'sm2-v1~) would decrease the ease factor by
0.15, then calculate the next interval by multiplying the previous
interval with the new ease factor.

In the new version (~'sm2-v2~), the interval is always multiplied by a
factor of 1.2, similar to the version of SM2 used by Anki.

~org-fc-algorithm~ can be used to set which version of the
algorithm should be used, defaulting to ~'sm2-v1~.

Once I have evaluated the performance of the new algorithm,
the default version will change to ~'sm2-v2~.
*** [2020-04-12 Sun]
**** Added
- =text-input= card type
*** [2020-02-08 Sat]
**** Changed
- Add a "Z" suffix to all ISO8601 timestamps
**** Added
- A function to estimate the number of reviews in the next n days
*** [2020-02-03 Mon]
**** Internal
- ~org-fc-due-positions-for-paths~ now shuffles the lists of positions
  using an Emacs Lisp function instead of depending on =shuf=
- All awk-indexer functions now use ~gawk~ instead of ~awk~
** Other Documentation
- [[file:doc/internals.org][Internals]]
- [[file:doc/sharing_decks.org][Sharing Decks]] (todo)
** License
Copyright © Leon Rische and contributors. Distributed under the GNU General Public License, Version 3