Swiftpack.co - Package - deepmind/open_spiel
Swiftpack.co is a collection of thousands of indexed Swift packages. Search packages.

OpenSpiel: A Framework for Reinforcement Learning in Games

Documentation Status Build Status

OpenSpiel is a collection of environments and algorithms for research in general reinforcement learning and search/planning in games. OpenSpiel supports n-player (single- and multi- agent) zero-sum, cooperative and general-sum, one-shot and sequential, strictly turn-taking and simultaneous-move, perfect and imperfect information games, as well as traditional multiagent environments such as (partially- and fully- observable) grid worlds and social dilemmas. OpenSpiel also includes tools to analyze learning dynamics and other common evaluation metrics. Games are represented as procedural extensive-form games, with some natural extensions. The core API and games are implemented in C++ and exposed to Python. Algorithms and tools are written both in C++ and Python. There is also a branch of pure Swift in the swift subdirectory.

To try OpenSpiel in Google Colaboratory, please refer to open_spiel/colabs subdirectory or start here.

OpenSpiel visual asset

Index

Please choose among the following options:

For a longer introduction to the core concepts, formalisms, and terminology, including an overview of the algorithms and some results, please see OpenSpiel: A Framework for Reinforcement Learning in Games.

For an overview of OpenSpiel and example uses of the core API, see the tutorial presentation slides: Introduction to OpenSpiel.

If you use OpenSpiel in your research, please cite the paper using the following BibTeX:

@article{LanctotEtAl2019OpenSpiel,
  title     = {{OpenSpiel}: A Framework for Reinforcement Learning in Games},
  author    = {Marc Lanctot and Edward Lockhart and Jean-Baptiste Lespiau and
               Vinicius Zambaldi and Satyaki Upadhyay and Julien P\'{e}rolat and
               Sriram Srinivasan and Finbarr Timbers and Karl Tuyls and
               Shayegan Omidshafiei and Daniel Hennes and Dustin Morrill and
               Paul Muller and Timo Ewalds and Ryan Faulkner and J\'{a}nos Kram\'{a}r
               and Bart De Vylder and Brennan Saeta and James Bradbury and David Ding
               and Sebastian Borgeaud and Matthew Lai and Julian Schrittwieser and
               Thomas Anthony and Edward Hughes and Ivo Danihelka and Jonah Ryan-Davis},
  year      = {2019},
  eprint    = {1908.09453},
  archivePrefix = {arXiv},
  primaryClass = {cs.LG},
  journal   = {CoRR},
  volume    = {abs/1908.09453},
  url       = {http://arxiv.org/abs/1908.09453},
}

Versioning

We use Semantic Versioning

Github

link
Stars: 2549

Dependencies

Releases

OpenSpiel 0.2.0 - 2020-12-07T09:54:54

This is our second release, mainly a stable version to use for our first entry in pip/PyPI.

Changes since last release:

Availability in PyPI as a source distribtuion. See installation page for instructions.

New Games

  • Battleship
  • Blackjack
  • Clobber
  • Hearts
  • Klondike Solitaire
  • Lewis Signaling
  • Oh Hell
  • Sheriff
  • Slovenian Tarok
  • Stones & Gems

New Algorithms

  • CorrDist correlated equilibrium gap metrics for agent-form (AFCE, AFCCE)
  • DQN (PyTorch)
  • Deep CFR (TF2)
  • External Sampling MCCFR (Python)
  • Online Outcome Sampling (Lisy et al, 2015)
  • NFSP (PyTorch)
  • Policy Iteration
  • Regression CFR (PyTorch) (Waugh et al. 2015, Morrill 2016)
  • ResponseGraphUCB (Rowland et al. 2019)

New Support

  • Build as a shared library using BUILD_SHARED_LIB
  • Eigen
  • Libtorch (PyTorch's C++ API)
  • PyTorch (including 3 algorithms)

Bots

  • Roshambo (rock paper scissors competition bots), optional using BUILD_WITH_ROSHAMBO
  • xinxin (Nathan Sturtevant's Hearts bot), optional using BUILD_WITH_XINIXIN

Misc

  • C++ LP solving via OR-Tools
  • Correlated equilibrium solver via LP solver (C++)
  • HyperGammon option to Backgammon
  • Exact correlation device builder (for extensive-form correlated equilibria)
  • Gambit normal-form game (.nfg) writer and reader
  • Game Transform to start at a specific place in a game
  • Observation framework
  • Public State API and factored observation games (FOGs): public observations, action-observation histories, visualizations (Kovarik et al.)
  • Repeated games
  • Single-player turns in simultaneous move games
  • Serialization and checkpointing in CFR, MCCFR, CFR-BR
  • Serialization for kSampledStochastic chance modes
  • Suggested alpha function for alpha-rank
  • Supervised learning example for Hearts (based on xinxin self-play)

New utilities

  • ActionView
  • Random interface
  • down_cast
  • combinatorial helpers

New functionality, but untested or not fully supported

  • C++ AlphaZero via TensorflowCC
  • GAMUT integration

Removals

  • dependency on sonnet (due to clashes with TF1; our uses replaced by simple_nets.py)
  • games are no longer copyable
  • PSRO version 1

Acknowledgments

Many thanks to DeepMind for their continued support of OpenSpiel.

Thank you to all of our contributors:

Files

  • The open_spiel-0.2.0.tar.gz and open_spiel-0.2.0.zip archives below include minimal dependencies necessary to build OpenSpiel, including some optional games, such as the Hanabi Learning Environment and universal_poker wrapper to ACPC poker code.

OpenSpiel 0.1.0 - 2020-06-11T20:41:46

Release 0.1.0

This is our first release since we initially released OpenSpiel in August 2019.

The main purpose is to provide a stable version for anybody currently using the code base for research, or any other form of dependency, as we are planning a number of changes that could be disruptive. Please see #166 and #238 for details.

Changes since initial release

General support:

  • MacOS
  • Windows (via Subsystem for Linux)
  • Docker install
  • C++17 (from C++11)
  • Clang (from g++)
  • Julia API
  • Colab
  • pip installable package

New Algorithms

  • AlphaZero (Python, C++)
  • CFR-BR (C++, Python)
  • CorrDist (analogues to NashConv for correlated equilibria, C++)
  • Discounted CFR (Python)
  • Double-Oracle solver
  • General-sum Nash solver, iterated dominance
  • Information Set MCTS (IS-MCTS)
  • Linear CFR (Python)
  • Outcome Sampling MCCFR (C++)
  • PSRO v2 (with Deep RL Oracles)

New Games

  • Cliff Walking
  • Coordinated Matching Pennies
  • Cursor Go
  • Deep Sea
  • Gambit .efg parser
  • Gin Rummy
  • GridMaze (Swift)
  • Hanabi (via Hanabi Learning Environment)
  • Hold’em Poker (via ACPC wrapper)
  • Laser Tag
  • Negotiation
  • Othello
  • Quoridor
  • Skat
  • Texas Hold’em (Swift)
  • Tiny Hanabi
  • Trade Comm

Misc. functionality

  • Human player bot
  • TF and Batch Trajectories (fast C++ inference via tensorflow)
  • Player type, rather than int
  • Visualization of game trees via graphviz
  • Ludii wrapper
  • MCTS-Solver
  • Gradient clipping in policy gradients
  • Multiplayer MCTS
  • Serialization for games, states, game types
  • Python-only games
  • Utils library (json, logging, stats)
  • Gambit .efg writers (C++, Python)

Removed:

  • Python 2 support

In addition: many fixes.

Thank you to all our contributors!