Vision 2.0

Computer Vision based event organised by Robotics Club of IIT (BHU) Varanasi

Problem Statement

1. There are 2 paths in our arena (inner and outer square) with 4 connecting paths of different colours joining them.
   
2. Bot can change from outer path to inner path or vice versa. Bot is allowed to move in clockwise direction only.
3. The portion of the arena in black colour is restricted for the movement of the bot.
   
4. There will be 3 shapes (square, circle and triangle) of 2 different colours (red and yellow), distinguishing each block in 6 different ways.
   
5. On the outermost path there will be 4 black arrows at the end of connecting paths pointing in clockwise direction. These arrows mark the starting zone where the bot will be placed initially on any one of the arrows.
   
6. The centre of the arena is the home zone. The bot has to traverse the arena, complete a full round in a clockwise manner and finish at the home zone.
   

Task To Do

1. The bot is placed on one of the starting zones (represented by 4 black arrows).
   
2. An abbreviation which associate to specific colour and shape.
   

TR : Red Triangle

SR : Red Square

CR : Red Circle

CY : Yellow Circle

SY : Yellow Square

TY : Yellow Triangle

3. On start of each turn, a function returns a random shape-color combination from the list above.The bot must then find the closest block (with the corresponding shape) which it can reach following a clockwise path.
   
4. As soon as the bot stops moving, bot has to ask for input using the function provided.
   
5. This continues till the bot has completed a full round around the center, then it should move to home via the connecting paths that it started on.
   
6. On reaching home the bot should signal that it has finished the task.
   

Our Approach

1. We used Computer Vision for image segmentation i.e. extracting shapes of different colors from the arena. Used Breadth First Search algorithm (on a customly designed directed graph) to trace path from the current position of the bot to all occurences of the corresponding shape in the arena. From all the paths secured above, one with minimum length was considered. Popular physics engine PyBullet was utilized for simulating our bot on the arena. Aruco Marker was used to determine the current position of the bot at any instant.
   
2. First, a directed graph is created, in which edges are added in the direction of allowed movement.
   
3. Shape and Color in each grid of the arena is detected using several techniques such as masking, erosion, dilation and contour approximation.
   
4. A function (roll_dice in our case) returns a shape-color combination to the bot in order to figure out the next destination to visit.
   
5. Then, BFS (Breadth First Search) is used to determine the all possible paths from the current position to the next destination. Among the paths secured, one with minimum length is considered for traversal
   
6. Two vectors are created providing the positions, along with the angles, of the bot and the destination grid. Various custom-made functions, such as dist(), ang(), rotate() and move(), are employed in order to facilitate the movement of the bot.
   
7. After the bot crosses the first grid, the graph edges are altered in a way that the bot enters the home after completion of a clockwise round and doesn't retrace the previous path.The task is completed after the bot reaches the central home grid.
   
8. Video of the run is provided in the repository