DIY Portable Face Recogniser Python Part 4

April 10, 2020 funincomplete Artificial Inteligence, Raspberry Pi, Technology 0

Main Program Loop

Okay, we will enter the main loop of the program in this post.

The first part of the loop has:

  • Prepare to catch keyboard interruption
  • Start the main program loop
  • Reset default values for each iteration
### Main program loop
try:
	while(endProgramCount  <  endProgramAfter):
		sleepTimer = 0
		NewSpeech = ''
		msg1 = format(WeekDay[datetime.now().weekday()]) + datetime.now().strftime(" %d-%b %H:%M")
		msg2 = 'REC          OFF'

Line 2: Prepare for KeyboardInterrupt exception
Line 3: Keep looping until the program shutdown sequence is triggered
Line 4-7: Reset variables for each iteration

Program Mode

We have different behavior for each program mode from here.

Detecting
  • Get an image from camera
  • Detect faces in the image
  • Recognise the detected faces using the face recognition model
  • Get the name of recognised face
  • Greet on LCD screen depending on the confidence level

The confidence values are:

  • 0 = 100% (Highest)
  • 100 = 0% (Lowest)
		if 'Detecting'==mode and not newTrainer:
			for i in range(5):
				ret, frame = cap.read()
			frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
			faces = face_cascade.detectMultiScale(frame, 1.3, 5)
			for (x,y,w,h) in faces:
				msg1 = 'Face Detected!'
				id, confidence = recogniser.predict(frame[y:y+h, x:x+w])
				sql = '''SELECT name FROM people WHERE ROWID = ? '''
				c.execute(sql, (str(id),))
				if confidence < 55:
					if confidence < 20:
						msg1 = 'Hello'
						msg2 = c.fetchone()[0]
					elif confidence < 25:
						msg1 = 'How are you'
						msg2 = c.fetchone()[0] + '?'
					elif confidence < 30:
						msg1 = 'You must be'
						msg2 = c.fetchone()[0]
					elif confidence < 35:
						msg1 = 'How is it going'
						msg2 = c.fetchone()[0] + '?'
					elif confidence < 40:
						msg1 = 'Nice to see you'
						msg2 = c.fetchone()[0]
					elif confidence < 45:
						msg1 = 'I think you are'
						msg2 = c.fetchone()[0]
					elif confidence < 50:
						msg1 = 'You look like'
						msg2 = c.fetchone()[0]
					else:
						msg1 = 'You may be'
						msg2 = c.fetchone()[0] + '?'
					NewSpeech = msg1 + ' ' + msg2 
				elif confidence < 100:
					msg1 = str(round(100-confidence))+ '% similar to'
					msg2 = c.fetchone()[0]

Line 1: If the mode is in “Detecting” then execute lines 2-39
Line 2 & 3: Get 5th frame of captured video to avoid getting a similar image
Line 4: Converting image colour to grayscale for face detection
Line 5: Detect face within the image using detector
Line 6-8: Cut out a face from a frame of image and recognise it using the face recognition model
Line 9 & 10: Retrieve the name of the recognised face
Line 11-35: If the confidence level is more than 45% then prepare a greeting message depending on the confidence level
Line 37-39: If the confidence level is less than 46% then show a different message

Recording
  • Prepare the message to indicate process progression 
		elif 'Recording' == mode:
			msg1 = indicator + ' Recording...'
			msg2 = str(len(trainingImg)) + ' Recorded'

Line 1: If the mode is in “Recording” then execute lines 2 & 3
Line 2 & 3: Prepare the message to indicate process progression

Predicting
  • Prepare a text to confirm if the suggestion is correct
  • Prepare a message to indicate button function
  • Find similar faces from the face recognition model
  • Get the names for similar faces from the database
		elif 'Predicting' == mode:
			if len(predictedIDset) == 0:
				for Img in trainingImg:
					id, confidence = recogniser.predict(Img)
					predictedIDset.add(id)
			else:
				if not (predictedID in predictedIDset):
					for i in predictedIDset:
						predictedID = i
						sql = '''SELECT name FROM people WHERE ROWID = ? '''
						c.execute(sql, (str(predictedID),))
						predictedName = c.fetchone()[0]
						break
			msg1 = 'Are you ' + predictedName + '?'
			msg2 = '   Yes    No'

Line 1: If mode is in “Predicting” then execute lines 2-15
Line 2: If prediction has not been made
Line 3-5: Find all similar faces from the face recognition model and put them in the predicted ID set
Line 6: If the prediction is has been made
Line 7: Skip lines 8-12 if the current suggestion is still in the predicted ID set
Line 8-13: If the current suggestion is already removed from the predicted ID set, then nominate the next suggestion from predicted ID set with the name retrieved from the database
Line 14: Prepare a text to confirm if the suggestion is correct
Line 15: Prepare a message to indicate button function

Updating
  • Show text on LCD screen
  • Add new face records for the existing person in the training model
  • Reset variables
  • Back to “Detecting” mode
		elif 'Updating'== mode:
			lcd.lcd_display_string('Updating...'.ljust(16), 1, 0)
			lcd.lcd_display_string('Please wait.'.ljust(16), 2, 0)
			recogniser.update(trainingImg, np_array([predictedID]*len(trainingImg)))
			recogniser.write(ScriptLocation + '/face_recogniser.yml')
			resetPrediction()
			trainingImg = []
			mode = 'Detecting'

Line 1: If the program is in “Updating” mode then execute lines 2-8
Line 2 & 3: Show “Updating…” on the top row and “Please wait.” the bottom row of the LCD screen
Line 4: Update face recognition model by passing face images and database record ID to the “update” method
Line 5: Overwrite current face recognition model into “face_recogniser.yml”
Line 6 & 7: Reset variables used to update the face recognition model
Line 8: Switch back to “Detecting” mode

Naming
  • Prepare message to show current name entered
  • Prepare message to indicate button function
		elif 'Naming' == mode:
			if NameCursorBlink:
				msg1 = 'Name:'+name
				NameCursorBlink = False
			else:
				msg1 = 'Name:'+name+alphabet[alphabet_index]
				NameCursorBlink = True
			msg2 = 'Sve <   >  E   C'

Line 1: If the mode is in “Naming” mode then execute lines 2-8
Line 2-7: Prepare a text “Name:” followed by the entred name and blink next alphabet
Line 8: Prepare a text to indicate the button functions

Saving
  • Show text on LCD screen
  • Insert record into the database
  • Save a new face into the face training model
  • Reset some variables
  • Switch back to the “Detecting” mode 
		if 'Saving'==mode:
			lcd.lcd_display_string('Saving...'.ljust(16), 1, 0)
			lcd.lcd_display_string('Please wait.'.ljust(16), 2, 0)
		
			sql = '''INSERT INTO people(name, date_created) VALUES (?,?)'''
			c.execute(sql, (name, datetime.now()))
			conn.commit()
			rowid = c.lastrowid
			if newTrainer:
				recogniser.train(trainingImg, np_array([rowid]*len(trainingImg)))
			else:
				recogniser.update(trainingImg, np_array([rowid]*len(trainingImg)))
			print('write face_recogniser.yml')
			recogniser.write(ScriptLocation + '/face_recogniser.yml')
			newTrainer = False
			resetPrediction()
			trainingImg = []
			name = ''
			alphabet_index = 0
			mode = 'Detecting'

Line 1: If the program is in “Saving” mode then execute line 2-20
Line 2 & 3: Show “Saving…” on the top row and “Please wait.” the bottom row of the LCD screen
Line 5-7: Save the name into the database table called “people”
Line 8: Get the record ID for the name just entered from database
Line 9: If this is the first face to record
Line 10: Passing face images and database record ID to the “train” method to create the face recognition model
Line 11: If this isn’t the first face to record
Line 12: Passing face images and database record ID to the “update” method to add to the face recognition model
Line 14: Create or Overwrite “face_recogniser.yml” file using the current face recognition model
Line 15: Indicating that at least one face is recorded in the model
Line 16-19: Reset all variables used to save face
Line 20: Switch back to “Detecting” mode

Ending
  • Prepare a text indicating the program is shutting down
  • Prepare a text showing a number counting down
		elif 'Ending' == mode:
			msg1 = 'End Program in'
			msg2 = str(endProgramAfter - endProgramCount)

Line 1: If the mode is in “Ending” mode then execute lines 2 & 3
Line 2: Prepare a text indicating the program is shutting down
Line 3: Prepare a text showing a number counting down

Show Text on LCD Screen

		### Show message on LCD screen
		lcd.lcd_display_string(msg1.ljust(16), 1, 0)
		lcd.lcd_display_string(msg2.ljust(16), 2, 0)

Line 2: Show message on the top row of the LCD screen
Line 3: Show message on the bottom row of the LCD screen

Voice Output

  • Avoid repeating the same speech
  • Output voice
        ### Voice Output	
		if '' != NewSpeech and NewSpeech != OldSpeech:
			PicoSpeech(NewSpeech)
			OldSpeech = NewSpeech

Line 2: If there is a new speech and the new speech is different to previous speech then execute lines 3 & 4
Line 3: Output voice
Line 4: Remember what just has been said

This is the end of the main program loop.

Ending Program

The main program loop will end when:

  • User keep pressing the “OFF” button
  • Interrupted by a keyboard input such as Ctrl+C
### Out of main program loop 
	quit_program(True)
	
### Program interruption		
except KeyboardInterrupt:
	quit_program(False)

Line 2: Start quitting the program with device shutdown
Line 5-6: Start quitting the program on a keyboard interruption without device shutdown

What’s Next?

We will configure Raspberry pi to automatically start this program.