Debugging in Python with IDE


#1

Did anyone manage to debug in Python through an IDE like PyCharm?

At the bottom of the Best Practices there is a forum solution to debug in an IDE. The post is from the first competition of Halite and links to files that now exist at archive.
However, after following the instructions, the socket networking fails to relay info through the new halite engine.

I get the following error (truncated by terminal):

oleg@oleg-ThinkPad-S2:~/Documents/repos/halite2/dev$ ./halite -t -d "30 30" "python3 MyBot.py" "python3 pipe_socket_translator.py 2000"
Game constants: all default
python3 MyBot.py
python3 pipe_socket_translator.py 2000
Seed: 187872751 Dimensions: 30x30
Init Message sent to player 1.
Init Message sent to player 0.
Init Message received from player 1, Traceback (most recent call la.

Has anyone looked into that or do you have a simpler solution to debug Python code in an IDE akin to the one suggested for Java in this post?


#2

@lidavidm @harikmenon1 any thoughts on this?


#3

Hmm, it seems like that approach would still work. You should only need 3 calls to sendStringSocket in pipe_socket_translator.py, but you would have to rewrite socket_networking.py. You could probably take its sendString and getString and stick them in the Halite 2 starter kit, though. Let me give it a try.


#4

@okomarov, give this a try: https://gist.github.com/lidavidm/2611852878d92e252fa875d4c54efc46

In your bot, import socketnetworking and use socketnetworking.Game instead of hlt.Game.

If it works, we can update the website.


#5

Hi David,

Unfortunately, I can't make it work with PyCharm.

I am calling ./halite -t -d "240 160" "python3 MyBotDebug.py" "python3 botproxy.py" where the port number is now hardcoded to 2000 inside socketnetworking.py (relevant changes as follows):

import socket
import sys

import hlt

PORT_ = 2000

class Game(hlt.Game):
    def __init__(self, *args, **kwargs):
        self._buf = []
        self._connection = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self._connection.connect(("localhost", PORT_))
        super().__init__(*args, **kwargs)
...

and MyBotDebug.py is (relevant part):

# Setup remote debugging for PyCharm
import sys
sys.path.append('/Applications/PyCharm.app/Contents/debug-eggs/pycharm-debug-py3k.egg')
import pydevd
pydevd.settrace('localhost', port=socketnetworking.PORT_)

import socketnetworking
game = socketnetworking.Game("okoma_bot")

while True:
    game_map = game.update_map()
...

It gets stuck at line 39 of socketnewroking.py call c = self._connection.recv(1), that is, the connection is probably not configured properly, even though I can step through in debug mode until then.

The log reports:

ERRORED! Got Exception (if any): 
ERROR: Bot #0: Received invalid character 
  File "/Applications/PyCharm.app/Contents/debug-eggs/pycharm-debug-py3k.egg/_pydevd_bundle/pydevd_comm.py", 
  line 365, in _on_runF'. (at character 3.)
Input received from bot:
  File "/Applications/PyCharm.app/Contents/debug-eggs/pycharm-debug-py3k.egg/_pydevd_bundle/pydevd_comm.py", 
  line 365, in _on_run
Response received (if any):   
  File "/Applications/PyCharm.app/Contents/debug-eggs/pycharm-debug-py3k.egg/_pydevd_bundle/pydevd_comm.py", 
  line 365, in _on_run    
r = self.sock.recv(1024)
ConnectionResetError: [Errno 54] Connection reset by peer

So, probably something related to PyCharm (?). Anyone succeded with another IDE?


#6

Hmm. How are you running everything? First, outside of PyCharm, you should invoke Halite and type in the port you want to use. Inside PyCharm, you should run or debug your bot file itself.


#7

First, I start the debug server in PyCharm with address localhost:2000, then call ./halite -t -d "240 160" "python3 MyBotDebug.py" "python3 botproxy.py" where the port number is now hardcoded, since the input() was not working.

The details of the port hardcoding are above.


#8

I also get and "Address already in use" although I make sure to kill all python processes before re-running the halite command.

Log player 0

[
 {
  "Error": {
   "Message": "ERRORED! Got Exception (if any): ERROR: Bot #0: Received invalid character '  File \"MyBotDebug.py\", line 14, in <module>F'. (at character 3.)\nInput received from bot:\n  File \"MyBotDebug.py\", line 14, in <module>\n  ^; Response received (if any):   File \"MyBotDebug.py\", line 14, in <module>    game = socketnetworking.Game(\"okoma_bot\")  File \"/Users/okomarov/Documents/repos/halite2/socketnetworking.py\", line 13, in __init__    super().__init__(*args, **kwargs)  File \"/Users/okomarov/Documents/repos/halite2/hlt/networking.py\", line 78, in __init__    tag = int(self._get_string())  File \"/Users/okomarov/Documents/repos/halite2/socketnetworking.py\", line 39, in _get_string    c = self._connection.recv(1).decode(\"ascii\")ConnectionResetError: [Errno 54] Connection reset by peer",
   "Turn": 1
  },
  "Frames": [
   {
    "Time": 12336,
    "Turn": 0
   },
   {
    "Time": 0
   }
  ],
  "PlayerID": 0,
  "PlayerName": "Traceback (most recent call la"
 },
 {
  "Error": {
   "Message": "ERRORED! Got Exception (if any): ERROR: Bot #1: Received invalid character '  File \"botproxy.py\", line 7, in <module>F'. (at character 3.)\nInput received from bot:\n  File \"botproxy.py\", line 7, in <module>\n  ^; Response received (if any):   File \"botproxy.py\", line 7, in <module>    socket.bind(('localhost', PORT_))OSError: [Errno 48] Address already in use",
   "Turn": 1
  },
  "Frames": [
   {
    "Time": 58,
    "Turn": 0
   },
   {
    "Time": 0
   }
  ],
  "PlayerID": 1,
  "PlayerName": "Traceback (most recent call la"
 }
]

#9

Hmm, when I was testing it, it was the other way around - run halite first with the proxy (and also, run it versus a bot that is not using any of this debug stuff). Then, start the bot independently on its own.

# In one terminal
./halite -t -d "240 160" "python3 botproxy.py 1337"  "python3 null.py"
# Then, in a completely separate terminal
$ python3 -m pdb MyBot.py

#10

Hello,
I tried from Windows.
When I run in the anaconda console:
halite -t -d "240 160" "python botproxy.py 1337" "python OldBots/MyBot3.py"

I get this error:
Game constants: all default
Seed: 1492599611 Dimensions: 240x160
Init Message sent to player 0.
Init Message sent to player 1.
Init Message received from player 0, Traceback (most recent call la.
Init Message received from player 1, gavi_v03.
Turn 1
ERROR: Bot #0: Received invalid character ' File "botproxy.py", line 6, in F'. (at character 3.)
Input received from bot:
File "botproxy.py", line 6, in
^
Player 0 is dead
Skipping replay (bot errored on first turn).
Player #0, Traceback (most recent call la, came in rank #2 and was last alive on frame #0, producing 0 ships and dealing 0 damage!
Player #1, gavi_v03, came in rank #1 and was last alive on frame #1, producing 0 ships and dealing 0 damage!
Player 1 is dead

and line 6 of botproxy is like in your github:
socket.bind(('localhost', int(sys.argv[1])))

any advice on how to debug from Visual Studio or other IDE?
can you make a video or amore detailed explanation, please?


#11

Here is an example from the terminal: https://asciinema.org/a/qRpRTZRptJ63LNqJmBWaroZAa

It should be similar for Visual Studio et al, but I don't have Visual Studio to test with.

You should not test against the same bot you are trying to debug. Test against a different bot. You may also have to try a few different ports; it seems it doesn't properly close the socket on exit.


#13

thank you. It is different in my Windows environment.
In Windows, I can't do step 2 (running the bot) because step 1 (running the proxy) exits with a crash.
I appreciate your attempt.


#14

If you run the proxy by itself, outside of Halite, what does the crash say? That would help me a lot in figuring out what is going wrong. e.g. just python3 botproxy.py 12345


#15

You guys are gonna hate me, but I used to do for this was:

  1. Disable timeout
  2. Setup a loop in my code based on a variable value
  3. Start a match
  4. Attach to process
  5. Change the value through the debugger to start playing the match

It's shameful, I know, but it also worked.


#16

@lidavidm: sorry, after looking at the code, I understood. Your system works very well also in Windows.
(there is an error after halite.exe finishes the battle, but that is not important).

@ewirkerman: thank you, your way work well.