Turns out the problem was with 1Password extension for Chrome, once I disabled that the problem disappeared.