View Thread

HellBound Hackers | Computer General | Programming

Author

Handycap Computer Usage

techb
Member

Posts: 194
Location:
Joined: 15.02.09
Rank: Wiseman

Posted on 25-02-10 00:01

This is a program I wrote back in high school. The idea came from Johnny Lee and what he did with the WII mote (http://www.youtube.com/watch?v=QgKCrGvShZs). I found more resources for filtering infrared light so I could use my webcam instead of a WII mote. Turns out that blank film negatives filter infrared light, so I rigged a couple of layers to fit over my laptops webcam.
About a week or so after writing the program I was selected to do an in person survey with the super attendant of the schools along with a few other students. One of the students was a quadriplegic. The schools in my area are trying to implement technology in their curriculum and the handicapped student complained that the schools are trying to encourage the use of technology but he wasn’t able to use it.
After hearing him talked about not being able to use a computer since his accident I decided to show him my program and offered to make a set-up at his house so he could use a computer. I took an old hat and rigged an infrared led to it along with a wireless mouse; I extended the click button down to his mouth. He has been using this program for about a year now and is surfing the web, sending emails, and watching porn like the rest of the world.

Requirements:
=-=-=python (python.org)
=-=-=PIL (http://www.pythonware.com/products/pil/)
=-=-=VideoCapture (http://videocapture.sourceforge.net/)
=-=-=pygame (http://www.pygame.org/news.html)



#Webcam IR mouse control 1.6

#Tech B Sept. 9, 2009

#Works only with an IR filter on the webcam.

#To do this simply cut out the black part of a film

#negitave, and place two or three layers over the lens

#of the webcam.

######################################################







#----------imports and global variabl assignment------

######################################################

from VideoCapture import Device

import Image, sys, pygame

from ctypes import *

import ImageOps

from PIL import ImageEnhance

from pygame.locals import *

cam = Device()

user = windll.user32

res = (1440,900) #set to the resolution of the screen

pygame.init()

screen = pygame.display.set_mode((640,480))

pygame.display.set_caption('IR Mouse Control')

font = pygame.font.SysFont("Curier",26)



#---------------functions-----------------------

################################################

#Returns the xy cordinets of the ir dot.

#Doesn't return exact dot, only the first TRUE pixel value

def xy(im):

  imxy = im.getprojection()

  imx = imxy[0]

  imy = imxy[1]

  x = imx.index(1)

  y = imy.index(1)

  return (x,y)



#Returns size in pixels of the ir dot

#Was going to be used for click event

#No longer needed, but I think its a cool feature =]

def irint(im):

  xyi = 0

  irxy = im.getprojection()

  irx = irxy[0]

  iry = irxy[1]



  for i in irx:

    if i == 1:

      xyi +=1

  for i in iry:

    if i == 1:

      xyi += 1

  return xyi









#-----------Main loop--------------------

#########################################

while 1:

  

  for event in pygame.event.get():

    if event.type == pygame.QUIT: sys.exit()

    

  try:

    imt = cam.getImage()

    im = imt.resize(res)

    im = ImageOps.mirror(im)

    im1 = ImageEnhance.Contrast(imt).enhance(1.0)

    im1 = ImageEnhance.Brightness(imt).enhance(1.5)

    x,y = xy(im)

    xyi = irint(im1)

    user.SetCursorPos(x,y)

    irpix = font.render("IR intensity in pixels:" + str(xyi), True, (250,250,250))

    name = font.render('By Tech B', True, (250,250,250))

    web = font.render('Webcam IR mouse control : 1.0.1', True, (250,250,250))

    fil = font.render('Works only with an IR filter on the webcam lens', True, (250,250,250))

    im1 = pygame.image.frombuffer(im1.tostring(), (640,480), "RGB") 

    screen.blit(im1, (0,0))

    screen.blit(name,(0,26))

    screen.blit(web,(0,0))

    screen.blit(fil,(0,52))

    screen.blit(irpix,(0,78))

    pygame.display.flip()

                      

  except ValueError:                   

    pass

Like I stated before the clicks are with a hacked mouse. I tried doing the click with code, but couldn't find a reliable solution.
Some draw backs include CPU usage is high and other infrared light can effect the mouse such as a cigarette or open flames.

The next step is to lose the IR filter and base it on facial recognition. I've seen it done, but don't quite know where to start.
Any suggestions or criticism is welcome, and feel free to tweak and distribute the code.

Author

RE: Handycap Computer Usage

wolfmankurd
Member

Posts: 1492
Location: UK
Joined: 30.05.05
Rank: God

Posted on 25-02-10 01:23

If I understand this correctly and he is indeed using your computer/hardware to do this shizzle. Then seriously top notch stuff. But I didn't read through it all mainly cause it's like 2 am.

BY READING MY POST, YOU ACCEPT IT AS IS AND AGREE TO MY DISCLAIMER OF ALL WARRANTIES, EXPRESS OR IMPLIED, AS WELL AS DISCLAIMERS OF ALL LIABILITY, DIRECT, INDIRECT, CONSEQUENTIAL OR INCIDENTAL, THAT MAY ARISE FROM THE USE OF THIS (MIS)INFORMATION.

Author

RE: Handycap Computer Usage

define
Member

Posts: 196
Location:
Joined: 13.12.08
Rank: Moderate
Warn Level: 1

Posted on 25-02-10 01:34

wolfmankurd wrote:
If I understand this correctly and he is indeed using your computer/hardware to do this shizzle. Then seriously top notch stuff.

I agree. The experiment on its own is of interest. The fact that it also can be put to good use is a plus.

I hopped on ye olde search engine and found a number of results but, since I am not more than a (barely) Python dabbler, I can't judge the usefulness of the resources. However, it looks like there are a couple of Python libraries for face recognition, and there's also this blog post that is, if not what you're looking for, full of links that give you more places to look.

I'd love to hear how further experimentation goes, too.

Yeah.

Author

RE: Handycap Computer Usage

stealth-
Ninja Extreme

Posts: 527
Location: Eh?
Joined: 10.04.09
Rank: God

Posted on 25-02-10 02:52

Very cool! This is a *great* example of how powerful python can be in the right hands

You should definitely let us know how any new experimentation works out.

The irony of man's condition is that the deepest need is to be free of the anxiety of death and annihilation; but it is life itself which awakens it, and so we must shrink from being fully alive.
www.stealth-x.com

Edited by stealth- on 25-02-10 02:52

Author

RE: Handycap Computer Usage

Compromise
Member

Posts: 224
Location:
Joined: 11.11.09
Rank: Moderate
Warn Level: 30

Posted on 25-02-10 08:57

It's a heart-warming thought that some people in this world can (and will) help people with some code, a bit of hardware and an old hat.

I'd be very interested in seeing a couple of pictures of the aforementioned hat, if you have any.

Rock on!

CrazySpai

Edited by Compromise on 25-02-10 09:02

Author

RE: Handycap Computer Usage

techb
Member

Posts: 194
Location:
Joined: 15.02.09
Rank: Wiseman

Posted on 25-02-10 20:50

I don't have any pictures of it. But I can describe it.
Its a black hat that has spots of white paint on the side. The IR led is in the center of the bill on the very edge. there is a switch from the battery back on the right side of the hat for the led. The mouse is velcroed on the other side with the face off of it. And a big gauge wire leading from the left mouse button down to the mouth. I used a push button switch for his button, its wrapped in shrink wrap to keep the saliva off of it.

total cost with out the mouse, $15.
with the mouse $45.

I had most of the items already though. I used to mess around with DC circuits and microcontrollers. BasicStamp2.

And thanks for the links.

Author

RE: Handycap Computer Usage

techb
Member

Posts: 194
Location:
Joined: 15.02.09
Rank: Wiseman

Posted on 25-02-10 21:06

Yeah lol. I got a kick out of micky lol.

Author

RE: Handycap Computer Usage

techb
Member

Posts: 194
Location:
Joined: 15.02.09
Rank: Wiseman

Posted on 25-02-10 21:31

Thanks.

Author

RE: Handycap Computer Usage

techb
Member

Posts: 194
Location:
Joined: 15.02.09
Rank: Wiseman

Posted on 25-02-10 22:08

UPDATE:

I can now read two IR dots. I'm thinking about basing the mouse movements on the slope of the two dots.



#hopeit with two IR points

#mouse control proto

#Tech B.

#d.bmp and d2.bmp are two 7X5 images to show the two

#  seprate points.





import VideoCapture

from VideoCapture import Device

import Image, sys, pygame, time

from pygame.locals import *

from psyco import full

from ctypes import *

user = windll.user32



full()



cam = Device()

pygame.init()

screen = pygame.display.set_mode((640,480))

font = pygame.font.SysFont("Curier",26)

d = pygame.image.load('d.bmp')

dT = pygame.image.load('d2.bmp')





def xy(im):

    imxy = []

    x = 0

    y = 0

    while x*y <= 307200: #Resolution of the cam. 640*480

        if im.getpixel((x,y)) >= (250,250,250):

            imxy.append((x,y))

        x += 1

        if x == 640:

            x = 0

            y += 1

            

        if y == 480:

          return imxy



x = 0

y = 0



while 1:

  try:

      

    for event in pygame.event.get():

        if event.type == pygame.QUIT: sys.exit()

    imc = cam.getImage()

    bg = pygame.Surface(screen.get_size())

    bg = bg.convert()

    bg.fill((0,0,0))

    dots = xy(imc)



    if dots:

        imc = pygame.image.frombuffer(imc.tostring(), (640,480), "RGB")

        bg.blit(imc, (0,0))

        bg.blit(d, dots[0])

        bg.blit(d, dots[-1])

        screen.blit(bg, (0,0))

        pygame.draw.line(screen,(255,255,255), dots[0], dots[-1])

        pygame.display.flip()



    else:

        pass

  except IndexError:

      pass