Build an autonomous agent

Describe how your agent interacts with the world by specifying its states, actions, transition function, and reward function. Select the classic grid world agent or make your own.

const agent = new Agent(...)

Choose an ethical system

Indicate how your agent reasons about the implications of its behavior by specifying its ethical context and moral principle. Select a traditional ethical system or make your own.

const ethics = new Ethics(...)

Bring them together

Generate a policy for your agent that allows it to complete its task as efficiently as possible while still following its ethical system. Say goodbye to reward hacking.

const solution = morality.solve(...)

Tutorial

Morality.js is a library for building autonomous agents that comply with ethical theories. Let's go through an example on how to build a simple grid world agent that follows different ethical systems with Morality.js. You can add Morality.js to your node project like any other npm package by running the command below.


  npm install --save morality

First, we have to build an autonomous agent. Suppose we have a robot searching for money in a grid world. The robot can stay in its current square or move to another square by going north, east, south, or west. However, when the robot tries to move to another square, it has a small chance of slipping into a square to the left or the right of its current square because one of its wheels can get stuck at times. In Morality.js, we can define a grid world agent using an array where the symbol O represents an empty square, the symbol W represents a wall square, and the symbol G represents a goal square.


  const agent = new GridWorldAgent([
    ['O', 'O', 'W', 'W', 'O'],
    ['O', 'O', 'W', 'W', 'O'],
    ['O', 'O', 'O', 'O', 'G']
  ]);

Next, if we want our robot to comply with an ethical theory, we have to choose an ethical system. Morality.js offers a few ethical systems right out of the box: divine command theory, prima facie duties, and virtue ethics. Divine command theory is a simple ethical system that says that an agent should do whatever is commanded or forbade by a divine entity. In other words, the agent can't enter certain states of the world. In Morality.js, we can define divine command theory by listing each forbidden state.


  const ethics = new DivineCommandTheory([0, 4, 10]);

Prima facie duties is a more sophisticated ethical system that says that an agent should follow duties. To do this, we need a few components: a set of duties, a violation function, a penalty function, and a tolerance. First, the set of duties represents the duties that must be followed by the agent. Next, the violation function represents whether or not each state neglects a duty. Moreover, the penalty function represents the cost of neglecting a duty given that the agent performed a specific action in a particular state. Finally, the tolerance represents the slack that is given to the agent. In Morality.js, we can define prima facie duties by specifying each of these components.


  const duties = ['Quiet Operation', 'Personal Space'];

  const violationFunction = (state) => {
    if (state == 6) {
      return ['Quiet Operation', 'Personal Space'];
    }
    return [];
  };

  const penaltyFunction = (duty, state) => {
    if (duty == 'Quiet Operation') {
      return 1;
    }
    if (duty == 'Personal Space') {
      return 10;
    }
    return 0;
  };

  const tolerance = 0.3;

  const ethics = new PrimaFacieDuties(duties, violationFunction, penaltyFunction, tolerance);

Virtue ethics is an ethical system that says that an agent should strive to behave like a moral exemplar. To do this, the agent should perform any action that would be performed by a moral exemplar in a specific state. This means we need a list of trajectories that have been experienced by moral exemplars. Each trajectory has a list of the states that have been encountered and a list of actions that have been performed by a moral exemplar. In Morality.js, we can define virtue ethics by specifying this list.


  const ethics = new VirtueEthics([
    [[0, 1], ['EAST', 'SOUTH']], 
    [[5, 6], ['SOUTH'], ['EAST']]
  ]);

Finally, once we've defined an autonomous agent and an ethical system, we can bring them together by using Morality.js.


  const solution = morality.solve(agent, ethics);

A solution in Morality.js is just a map with a value function linking each state to its optimal value and a policy linking each state to its optimal action.


  {
    'values': {
      '0': 84.9,
      '1': 85.5,
      ...
    },
    'policy': {
      '0': 'SOUTH',
      '1': 'EAST', 
      ...
    }
  }

Playground

Curious? Play around with a simple grid world that follows different ethical systems.

Make sure your browser window is large enough to check out the playground.

Citation

Please cite the following paper if you use Morality.js in your own research.

  • Ethically Compliant Sequential Decision Making
  • Justin Svegliato — Samer B. Nashed — Shlomo Zilberstein
  • Proceedings of the 35th Conference on Artificial Intelligence (AAAI)
  • Expected March 2021