Introduction

This document describes the legacy TCP API of VNDB. Usage of this TCP API in new applications in discouraged, refer to the new HTTPS API for a more modern replacement.

Usage terms

This service is free for non-commercial use. The API is provided on a best-effort basis, no guarantees are made about the stability or applicability of this service.

The data obtained through this API is subject to our Data License.

Design goals

• Simple in implementation of both client and server. “Simple” here means that it shouldn’t take much code to write a secure and full implementation and that client applications shouldn’t require huge dependency trees just to use this API.
• Powerful: Not as powerful as raw SQL, but not as rigid as commonly used REST or RPC protocols.
• High-level: common applications need to perform only few actions to get what they want.
• Fast: minimal bandwidth overhead and simple and customizable queries.

Design overview

• TCP-based, all communication between the client and the server is done using one TCP connection. This connection stays alive until it is explicitely closed by either the client or the server.
• Request/response, client sends a request and server replies with a response.
• Session-based: clients are required to login before issuing commands to the server. A session is created by issuing the ‘login’ command, this session stays valid for the lifetime of the TCP connection.
• Everything sent between the client and the server is encoded in UTF-8.

Limits

The following limits are enforced by the server, in order to limit the server resources and prevent abuse of this service.

• 10 connections per IP. All connections that are opened after reaching this limit will be immediately closed.
• 200 commands per 10 minutes per ip. Server will reply with a ‘throttled’ error (type=“cmd”) when reaching this limit.
• 1 second of SQL time per minute per ip. SQL time is the total time taken to run the database queries for each command. This depends on both the command (filters and get flags) and server load, and is thus not very predictable. Server will reply with a ‘throttled’ error with type=“sql” upon reaching this limit.
• Each command returns at most 25 results, with the exception of get votelist/vnlist/wishlist/ulist, which returns at most 100 results.

These limits may sound strict, but in practice you won’t have to worry much about it. As long as your application properly waits when the server replies with a “throttle” error, everything will be handled automatically. In the event that your application does require more resources, don’t hesitate to ask.

Connection info:

Host

api.vndb.org

Port (plain tcp)

19534 (‘VN’)

Port (TLS)

19535

For improved security, make sure to verify that the certificate is valid for ‘api.vndb.org’ and is signed by a trusted root (in particular, by Let’s Encrypt).

Request/response syntax

The VNDB API uses the JSON format for data in various places, this document assumes you are familiar with it. See JSON.org for a quick overview and RFC 4627 for the glory details.

The words object, array, value, string, number and integer refer to the JSON data types. In addition the following definitions are used in this document:

request or command

Message sent from the client to the server.

response

Message sent from the server to the client.

whitespace

Any sequence of the following characters: space, tab, line feed and carriage return. (hexadecimal: 20, 09, 0A, 0D, respectively). This is in line with the definition of whitespace in the JSON specification.

date

A string signifying a date (in particular: release date). The following formats are used: “yyyy” (when day and month are unknown), “yyyy-mm” (when day is unknown) “yyyy-mm-dd”, and “tba” (To Be Announced). If the year is not known and the date is not “tba”, the special value null is used.

Message format

A message is formatted as a command or response name, followed by any number of arguments, followed by the End Of Transmission character (04 in hexadecimal). Arguments are separated by one or more whitespace characters, and any sequence of whitespace characters is allowed before and after the message.

The command or response name is an unescaped string containing only lowercase alphabetical ASCII characters, and indicates what kind of command or response this message contains.

An argument can either be an unescaped string (not containing whitespace), any JSON value, or a filter string. The following two examples demonstrate a ‘login’ command, with an object as argument. Both messages are equivalent, as the whitespace is ignored. ‘0x04’ is used to indicate the End Of Transmission character.

login {"protocol":1,"username":"ayo"}0x04

login {
 "protocol" : 1,
 "username" : "ayo"
}
0x04

The 0x04 byte will be ommitted in the other examples in this document. It is however still required.

Filter string syntax

Some commands accept a filter string as argument. This argument is formatted similar to boolean expressions in most programming languages. A filter consists of one or more expressions, separated by the boolean operators “and” or “or” (lowercase). Each filter expression can be surrounded by parentheses to indicate precedence, the filter argument itself must be surrounded by parentheses.

An expression consists of a field name, followed by an operator and a value. The field name must consist entirely of lowercase alphanumeric characters and can also contain an underscore. The operator must be one of the following characters: =, !=, <, <=, >, >= or ~. The value can be any valid JSON value. Whitespace characters are allowed, but not required, between all expressions, field names, operators and values.

The following two filters are equivalent:

 (title~"osananajimi"or(id=2))

 (
   id = 2
   or
   title ~ "osananajimi"
 )

More complex filters are also possible:

 ((platforms = ["win", "ps2"] or languages = "ja") and released > "2009-01-10")

See the individual commands for more details.

The ‘login’ command

 login {"protocol":1,"client":"test","clientver":0.1,"username":"ayo","password":"hi-mi-tsu!"}

Every client is required to login before issuing other commands. The login command accepts a JSON object as argument. This object has the following members:

protocol

An integer that indicates which protocol version the client implements. Must be 1.

client

A string identifying the client application. Between the 3 and 50 characters, must contain only alphanumeric ASCII characters, space, underscore and hyphens. When writing a client, think of a funny (unique) name and hardcode it into your application.

clientver

A number or string indicating the software version of the client.

username

(optional) String containing the username of the person using the client. When this field is provided, the client must also provide either a “password” or “sessiontoken”.

password

(optional) String, password of that user in plain text.

sessiontoken

(optional) String, to log in with a session token instead of password.

createsession

(optional) Boolean, only available when logging in with a password. This will create a new session token so that future logins can be done with the “sessiontoken” field instead of providing a password.

The server can reply with one of the following responses:

ok

No arguments, returned when the login command is successful and “createsession” was not specified.

session

Returned when the login is successful and the “createsession” field was specified. The response has one argument: the session token encoded as a hex string. The token will automatically expire one month after its last use, when the ‘logout’ command is used (see below) or when the user changes their password.

error

Login failed, see below for error codes.

Note that logging in using a username is optional, but some commands are only available when logged in. It is strongly recommended to connect with TLS when logging into an account.

Example login request and response without authentication:

 login {"protocol":1,"client":"Awesome Client","clientver":"1.0"}

 ok

Example login to obtain a session token:

 login {"protocol":1,"client":"Awesome Client","clientver":"1.0","username":"ayo","password":"xyz","createsession":true}

 session df0cc97e1f0c9f1d59ab67d2be3bb1d437892505

Later connections can use that token to log in:

 login {"protocol":1,"client":"Awesome Client","clientver":"1.0","username":"ayo","sessiontoken":"df0cc97e1f0c9f1d59ab67d2be3bb1d437892505"}

 ok

logout

When logged in with a session (either by specifying “createsession” or “sessiontoken” in the login command), the client can invalidate the token associated with the session by sending the ‘logout’ command without arguments:

 logout

The server will respond with ‘ok’ and disconnect.

The ‘dbstats’ command

This command gives the global database statistics that are visible in the main menu of the site. The command is simply:

 dbstats

And the response has the following format:

 dbstats stats

Where stats is a JSON object with integer values. Example response:

 dbstats {"users":0,
          "threads":0,
          "tags":1627,
          "releases":28071,
          "producers":3456,
          "chars":14046,
          "posts":0,
          "vn":13051,
          "traits":1272}

The users, threads and posts stats are always ‘0’ and only included for backwards compatibility.

The ‘get’ command

This command is used to fetch data from the database. It accepts 4 arguments: the type of data to fetch (e.g. visual novels or producers), what part of that data to fetch (e.g. only the VN titles, or the descriptions and relations as well), a filter expression, and lastly some options.

 get type flags filters options

type and flags are unescaped strings. The accepted values for type are documented below. flags is a comma-separated list of flags indicating what info to fetch. The filters, available flags and their meaning are documented separately for each type. The last options argument is optional, and influences the behaviour of the returned results. When present, options should be a JSON object with the following members (all are optional):

page

integer, used for pagination. Page 1 (the default) returns the first 10 results (1-10), page 2 returns the following 10 (11-20), etc. (The actual number of results per page can be set with the “results” option below).

results

integer, maximum number of results to return. Also affects the “page” option above. For example: with “page” set to 2 and “results” set to 5, the second five results (that is, results 6-10) will be returned. Default: 10.

sort

string, the field to order the results by. The accepted field names differ per type, the default sort field is the ID of the database entry.

reverse

boolean, default false. Set to true to reverse the order of the results.

The following example will fetch basic information and information about the related anime of the visual novel with id = 17:

 get vn basic,anime (id = 17)

The server will reply with a ‘results’ message, this message is followed by a JSON object describing the results. This object has three members: ‘num’, which is an integer indicating the number of results returned, ‘more’, which is true when there are more results available (i.e. increasing the page option described above will give new results) and ‘items’, which contains the results as an array of objects. For example, the server could reply to the previous command with the following message:

 results {"num":1, "more":false, "items":[{
   "id": 17, "title": "Ever17 -the out of infinity-", "original": null,
   "released": "2002-08-29", "languages": ["en","ja","ru","zh"],
   "platforms": ["drc","ps2","psp","win"],"anime": []
 }]}

Note that the actual result from the server can (and likely will) be formatted differently and that the order of the members may not be the same. What each member means and what possible values they can have differs per type and is documented below.

get vn

The following members are returned from a ‘get vn’ command:

Sorting is possible on the following fields: id, title, released, popularity, rating, votecount.

‘get vn’ accepts the following filter expressions:

get release

Returned members:

Sorting is possible on the ‘id’, ‘title’ and ‘released’ fields.

Accepted filters:

get producer

Returned members:

Sorting is possible on the ‘id’ and ‘name’ fields.

The following filters are recognised:

get character

Returned members:

Sorting is possible on the ‘id’ and ‘name’ fields.

The following filters are recognised:

get staff

Unlike other database entries, staff have more than one unique identifier.

There is the main ‘staff ID’, which uniquely identifies a person and is what the staff pages on the site represent.

Additionally, every staff name and alias also has its own unique identifier, which is referenced from other database entries to identify which alias was used. This identifier is generally hidden on the site and aliases do not have their own page, but the IDs are exposed in this API in order to facilitate linking between VNs/characters and staff.

Returned members:

Sorting is possible on the ‘id’ field.

The following filters are recognised:

get quote

Returned members:

Sorting is possible on the ‘id’ and the pseudo ‘random’ field (default).

The following filters are recognised:

Note that a filter is required for all get commands, so to get a random quote, use:

get quote basic (id>=1) {"results":1}

get user

Returned members:

The returned list is always sorted on the ‘id’ field.

The following filters are recognised:

get ulist-labels

Fetch the labels for a user. Returned members:

The returned list is always sorted on the ‘id’ field.

The following filters are recognised:

Labels marked as private are only returned for the currently logged in user.

Label ids are local to the user, id < 10 are built-in labels and are the same for every user, id >= 10 or above are custom labels created by the user or a migration script.

get ulist

This command replaces the (obsolete and now undocumented) “get votelist”, “get vnlist” and “get wishlist” commands.

Returned members:

Sorting is possible on the following fields: uid, vn, added, lastmod, voted, vote.

The following filters are recognised:

The ‘set’ command

The set command can be used to modify stuff in the database. It can only be used when logged in as a user. The command has the following syntax:

 set type id fields

Here, type is similar to the type argument to the ‘get’ command, id is the (integer) identifier of the database entry to change, and fields is an object with the fields to set or modify. If the fields object is not present, the set command works as a ‘delete’. The interpretation of the id and fields arguments depend on the type, and are documented in the sections below.

But before that, let me present some examples to get a feel on what the previous paragraph meant. The following example adds a ‘10’ vote on v17, or changes the vote to a 10 if a previous vote was already present:

 set ulist 17 {"vote":100}

And here’s how to remove Ever17 from the list:

 set ulist 17

‘set’ replies with a simple ‘ok’ on success, or with an ‘error’ (see below) on failure. Note that, due to my laziness, no error is currently returned if the identifier does not exist. So voting on a VN that does not exist will return an ‘ok’, but no vote is actually added. This behaviour may change in the future. Note that this API doesn’t care whether the VN has been deleted or not, so you can manage votes and stuff for deleted VNs (Which isn’t very helpful, because ‘get vn’ won’t return a thing for deleted VNs).

set ulist

This command replaces the “set votelist”, “set vnlist” and “set wishlist” commands.

This command facilitates adding, removing and modifying your VN list. The identifier argument is the visual novel ID, and the following fields are recognized:

When removing a ulist item, any releases associated with the VN will be removed from the users’ list as well. The release list functionality is not currently exposed to the API, so is only visible when the web interface is used.

The ‘error’ response

Every command to the server can receive an ‘error’ response, this response has one argument: a JSON object containing at least a member named “id”, which identifies the error, and a “msg”, which contains a human readable message explaining what went wrong. Other members are also possible, depending on the value of “id”. Example error message:

 error {"id":"parse", "msg":"Invalid command or argument"}

Note that the value of “msg” is not directly linked to the error identifier: the message explains what went wrong in more detail, there are several different messages for the same id. The following error identifiers are currently defined:

parse

Syntax error, unknown command or invalid argument type.

missing

A JSON object argument is missing a required member. The name of which is given in the additional “field” member.

badarg

A JSON value is of the wrong type or in the wrong format. The name of the incorrect field is given in a “field” member.

needlogin

Need to be logged in to issue this command.

throttled

You have used too many server resources within a short time, and need to wait a bit before sending the next command. The type of throttle is given in the “type” member, and the “minwait” and “fullwait” members tell you how long you need to wait before sending the next command and when you can start bursting again (this is the recommended waiting time), respectively. Both values are in seconds, with a precision of 0.1 seconds.

auth

(login) Incorrect username/password combination.

loggedin

(login) Already logged in. Only one successful login command can be issues on one connection.

gettype

(get) Unknown type argument to the ‘get’ command.

getinfo

(get) Unknown info flag to the ‘get’ command. The name of the unrecognised flag is given in an additional “flag” member.

filter

(get) Unknown filter field or invalid combination of field/operator/argument type. Includes three additional members: “field”, “op” and “value” of the incorrect expression.

settype

(set) Unknown type argument to the ‘set’ command.

Change Log

This section lists the changes made in each version of the VNDB code. Check out the announcements board for more information about updates.

2022-10-04

• Add “official” member to “get release basic”
• Add “id” and “thumbnail(|_width|height)” members to “get vn screens”
• Add “image_(width|height)” members to “get vn details”
• Add “image_(width|height)” members to “get character details”

2022-10-02

• Add “get vn titles”
• Add “length_minutes” and “length_votes” members to “get vn basic”
• Add “get release lang”
• Add “get release links”

2021-12-15

• Add support for creating and logging in with session tokens in the “login” command.

2021-11-15

• The “vn” object returned by “get release” now includes an “rtype” field.
• The “type” field returned by “get release” has been deprecated in favor of the above.

2021-01-30

• The “orig_lang” field in “get vn” now always returns exactly one language.

2020-12-29

• Add “get quote” command.

2020-11-13

• New fields for “get character”: age, cup_size and spoil_gender.

2020-07-09

• Deprecated the “image_nsfw” and “nsfw” flags given by “get vn details,screens”
• Added “image_flagging” fields to “get vn details” and “get character details”
• Added “flagging” field to “get vn screens”

2020-04-09

• The “dbstats” command no longer returns stats for users, threads and posts.

2020-01-01

• Deprecated the get/set votelist/wishlist/vnlists commands
• The “get ulist-labels”, “get ulist” and “set ulist” commands should now be used in new code
• See t13365 for more details

2019-12-05

• Early API support for the new lists feature. The votelist/wishlist/vnlist commands will be updated when it goes out of beta.
• Add “get ulist-labels”
• Add “get ulist”
• Add “set ulist”

2019-10-07

• Add wikidata links to “get vn/producer/staff”
• Add pixiv links to “get staff”

2018-06-13

• Add “get character instances”

2018-02-07

• The ‘aliases’ member for “get producer” is now uses newline as separation rather than a comma

2017-08-14

• Add ‘uid’ field to “get votelist/vnlist/wishlist” commands
• Add ‘vn’ filter to the same commands
• The ‘uid’ filter for these commands is now optional, making it possible to find all list entries for a particular VN

2017-06-21

• Add “resolution”, “voiced”, “animation” members to “get release” command
• Add “platforms” filter to “get release” command
• Accept arrays for the “vn” filter to the “get release” command
• Add “search” filter to “get staff”

2017-05-22

• Add “vns” and “voiced” flags to “get staff” command
• Add “voiced” flag to “get character” command

2017-04-28

• Add “get staff” command
• Add “staff” flag to “get vn” command

2.27

• Add “username” filter to “get user”
• Add “traits” filter to “get character”

2.25

• Add “tags” filter to “get vn”
• Increased connection limit per IP from 5 to 10
• Increased command limit from 100 to 200 commands per 10 minutes
• Added support for TLS
• Added “screens” flag and member to “get vn”
• Added “vns” flag and member to “get character”
• Allow sorting “get vn” on popularity, rating and votecount
• Added basic “get user” command
• Added “official” field to “get vn relations”

2.23

• Added new ‘dbstats’ command
• Added new ‘get’ types: character, votelist, vnlist and wishlist
• Added ‘set’ command, with types: votelist, vnlist and wishlist
• New error id: ‘settype’
• Added “tags” flag and member to “get vn”
• Added “stats” flag to “get vn”
• Added “firstchar” filter to “get vn”
• Added “vn” filter to “get character”

2.15

• Fixed a bug with the server not allowing extra whitespace after a “get ..” command
• Allow non-numbers as “clientver” for the login command
• Added “image_nsfw” member to “get vn”
• Added “results” option to the “get .. {}”
• Increased the maximum number of results for the “get ..” command to 25
• Added “orig_lang” member and filter to the “get vn ..” command
• Throttle the commands and sqltime per IP instead of per user
• Removed the limit on the number of open sessions per user
• Allow the API to be used without logging in with a username/password

2.12

• Added “image” member to “get vn”
• A few minor fixes in some error messages
• Switched to a different (and faster) search algorithm for “get vn .. (search ~ ..)”