How To
Make a vision-guided fireball-throwing catapult out of an ordinary industrial robot
Every hacker wants a budget to do bigger, cooler hacks. Well, we got our budget, all $1000 of it, and decided to turn a borrowed industrial robot into a catapult, a hack we'd been hoping to do for a long, long time. We'd been joking about throwing heavy objects with one of these robots ever since we saw the payload specs and an anvil in the shop.
We wanted to make a catapult that could destroy a car with bowling balls from at least 80 feet away, throw fireballs, and be controlled through a computer vision system so it could be aimed from a laptop. Result? Success.
Video 1, Video 2 at the bottom of this page
Taking the fireball photo
Throwing a fireball in the desert
It had been 117 degrees during the day, and this was not Wile E. Coyote desert, this was the desert that looked like a tinderbox with the additional kindling option. Somewhere out in that tinderbox of tumbleweeds and brush, my friends were pouring kerosene on a towel-covered bowling ball and loading it into a giant catapult. We had one fire extinguisher and a couple liters of water. If something caught on fire, we were screwed: the probably-survive-but-maybe-end-up-in-jail type of screwed. I briefly thought about how difficult this would be to explain to a judge.
This was hands-down the scariest photograph I've ever taken. Not only did the subject of the photograph have the potential to burn down California, but every picture was a 6-second exposure, so there were absolutely no second chances. It didn't help that we were wearing a little thin. We had only slept 9 hours in the past 3 days. We were confused from constant heat exposure. We were up at 3am trying to get this shot before the wind picked up. And we could hear one of the occupants of a nearby house screaming obscenities, presumably at us, while we assumed rattlesnakes were under every rock.
We had to reduce the distance the robot was throwing bowling balls because we didn't want the ball to go into the brush and catch the valley on fire. Also, the arc was just too enormous to get in a single photograph if we wanted any detail.
Framing the first fireball shot with LED tracers.
Framing the fireball shot
To frame the photograph, we taped LED lights to the bowling ball and threw it. It was just like one of those LED throwies, except that it was a bowling ball instead of a throwie, and it was a robot throwing it instead of a person; it was the desert and not a building in Venice, and this throwie could kill you. I took a picture, refocused, and continued to take photos, adjusting exposure every 20 seconds to compensate for the moon's movement.
Eli told me over the walkie that they were about to throw the fireball. I squeezed the shutter release halfway. Upon ignition, the fireball illuminated the robot and my frantic friends. I kept holding my breath and waited for the fireball to start moving, otherwise the exposure wouldn't catch the whole flight. As soon as the arm started to swing, I squeezed the shutter release. The resulting photo captured the fireball and if you look closely, Eli running at it with a penlight and a fire extinguisher. We didn't even burn down California. I call that a win.
Software
Custom software interface with grid option enabled
I wanted to be able to control the rotation of the robot so we could aim the robot from the laptop, but I quickly realized that since the desert is so flat, we could do some basic ranging on the target too. I also wanted the targeting to be overlaid in 3d over a photograph of the target area.
The software needed to control the robot like an MMO or RTS game. I suspect that video games, in general, have some of the most optimal control interfaces. I wanted to try a control scheme similar to the area effect spell targeting in World of Warcraft.
There were two pieces of information I needed the software to generate: the angle between the robot and the target and the range to the target. The former was very simple, and just required dividing the horizontal field of view by the pixels in the image. This calculation yielded a pixel-to-angle conversion from pixels in the image to angle between the camera, located at the base of the robot, and the target.
The range to the target was found in reverse. The user places the target in 3d on the plane where he or she wants the catapult to land a bowling ball. All that's necessary is to find the distance to the center of the target in the 3d coordinate space and then convert that to real-world coordinates.
The problem with the range calculation is that as the target approaches the horizon, the difference of a single pixel in the y-direction in the image can equal hundreds of feet in z-distance from the camera, or target range. In fact, with only a 4 ft separation between camera and ground plane, any range past 80 feet was unreliable.
I had thrown together a quick 3d viewer in VB.NET when I was working out 3d rotations for a machine vision application and decided it would be quicker to reuse that code than trying to remember my OpenGL and get it working on a .NET system. The 3d viewer was a nasty little hack, all done in GDI+, with the 3d projection taking a total of only 8 or 10 lines. After I built the target out of points and hooked up the arrow keys to move the target around, I had to accurately correlate the coordinate space in the 3d viewer to the real world. This involved learning the field of view in the vertical and horizontal directions on the camera and checking a measured point. Once the two coordinate spaces were correlated, I could just set up the virtual camera to match the real camera as being about 4 feet off the ground.
The serial ports are easy to use in VB .NET and I had the serial code for the Kuka left over from the WiiBot, so it was a cinch to hook the program to the Kuka robot to send the angle data and range data. I wrote a quick module in the Kuka in KRL to first move to the target angle, wait for a moment for the sling to stop swinging, and then fire. I used the range to scale the speed of the arm swinging through its launch arc. The few times I tried it in the shop were satisfactory. The hard part would be getting the range from the program to scale the speed of the arm so that the projectile would actually go the desired range.
Here is some of the KRL, the programming language the robot speaks:
The code above calls a function that reads in the serial, scales the speed, and then sets gposition.a1 to the proper target angle. The next lines just set up a movement structure and replace the a1-axis angle with the target angle for both the starting and end move. Each a-value correlates to one of the six axes on the robot. The ptp lines at the end are point-to-point moves and are the commands for the robot to throw the bowling ball.
When we got to the desert and actually threw some bowling balls I quickly realized two things:
• The range of the projectile scaled strangely, at best, and the robot threw past our 100' tape, so we would have limited data for a scaling function. For example, dropping the speed of the swing to 75% yielded about half of the distance of 100% power, but 87.5% power yielded about two-thirds of the distance of 100% power.
• I didn't bring a long enough usb cable so my laptop and I had to be within 3 feet of the base of robot to hit the serial send command. This wasn't going to happen; I was not going to chance bowling ball-induced death by a catapult that was over 26' tall at full swing.
Our solution was to abandon ranging, try targeting a few times by just manually punching in the numbers from the laptop into the controller. We finally just gave in to manually typing in the target angles into the robot, which made for some amusing trial and error.
Adjusting for maximum range
We all had our theories on how to get the maximum range out of the robot swing. Brian and I tried getting 5 of the axes to rotate at the same time (there are 6 axes on a Kuka). We tried giving it more snap in the wrist, the axis connected to the aluminum extrusion. We tried finishing so late in the swing the robot almost touched the ground. I even tried using the A2 axis exclusively to swing the robot (below).
One attempt at increasing the range of the bowling ball throw: not successful.
Once we had convinced ourselves we had the optimal motion path for the throw, we wanted even more distance (we were getting around 80 feet. I called the robot manufacturer to ask how to, uh, speed up the robot.
Me (Aaron) calling the robot manufacturer and somehow neglecting to mention that
we're speeding up the robot because we want it to throw bowling balls at an RV even harder.
A very helpful and knowledgable tech informed me that the payload setting could be reduced from the full 150kg to something more appropriate for our 6.3kg bowling ball. With that adjustment the robot started throwing bowling balls well past our 100' measuring tape, we're guessing about 120'.
Hardware:
Eli and Brian fix the release mechanism
Eli posing with an intimidating warning sign. I believe it translates into “Fighting with robot prohibited.”
Brian started construction by attaching a piece of aluminum extrusion to the robot, fabricating the sling, and testing the release mechanism. We tried to test the robot at the shop, but at full swing it was 26' tall and would've smashed through the ceiling of the shop.
We weren't sure exactly where to get a bunch of bowling balls for cheap, so we started with bowling alleys. One took pity on us and let us pick out an ugly bowling ball from their rack after we explained our project. As we were leaving, the woman at the counter said, “We'd better not see this on the news.” We found the other 5 at a bowling ball store in a bowling alley and they gave us 5 at $10 each instead of the usual $20 each.
All in all, it went pretty well. Our biggest snafus were a torn sling and a sheared bolt.
Eli tears the fraying sling with unnerving ease.
Fry's is awesome.
Release mechanism, classic medieval carabiners, bolts and aluminum extrusion.
Results:
Mid-swing
Bowling ball through windows
Watermelon hits RV
Don't watch this one if you have a weak stomach
Don't forget to buy some Mana Energy Potion!
Disclaimer: Harcos, Inc. is not affiliated with Kuka Robotics, in fact, we're not really sure what they're going to think about one of their robots being used for a catapult. Harcos, Inc. is also not affiliated with Pabst Blue Ribbon, it just happened to be cheap and we wanted to throw beer cans with a catapult.
