Installation

Download

Find a link for the latest beta version of xpcv on the Download page. Select your target platform and your preferred installation method.

Windows Setup Step 1

Download the installation file to your computer. When you’re ready to install the program, double-click the file and then follow the instructions on your screen. When opening an executable under Windows, you’ll will be probably shown a “Security Warning” screen. Click “Run” to start the setup wizard.

Installation using Windows Setup

Setup Wizard

Windows Setup Step 1

Follow the instructions of the xpcv Setup Wizard.
First, start the installation by clicking “Next” or cancel the installation by clicking “Cancel”.

Windows Setup Step 3

Accept the End-User License Agreement by reading the agreement carefully, marking the check box “I accept the terms in the License Agreement” (optionally printing out the agreement by clicking Print) and clicking “Next”, or go back to the previous step by clicking “Back” or cancel the installation process by clicking “Cancel”.

Windows Setup Step 4

Choose the destination folder for the installation of xpcv by clicking on “Browse…” or keep as is by clicking on “Next”, or go back to the previous step by clicking on “Back” or cancel the installation process by clicking on “Cancel”.

Windows Setup Step 5

In order to finish the installation of xpcv click on “Install”, or go back to the previous step by clicking on “Back” or cancel the installation process by clicking on “Cancel”.

Windows Setup Step 6

You’ll probably be asked for permission by the User Account Control (UAC) at this step. If you’re not logged in as administrator use the screen to log in. Click “Yes” to start the installation.

Windows Setup Step 8

Wait a few seconds for the installation being finished.
Windows Setup Step 9

The installation is finished now and the application is ready for the first start. Click on Finish for leaving the Setup Wizard.

First Start

Windows Setup Step 10

To start xpcv you can use Windows Search by opening the “Start” menu and then typing “xpcv”. Press “Enter” or click on “xpcv Gui” to start the graphical user interface.

Windows Setup Step 11

Alternatively you can navigate to the install folder by using the Windows Explorer. Default install directory: C:\Program Files\xpcv\

You’ll find the executable xpcv_gui.exe in the bin subfolder. Double click on the file to start xpcv.

Windows Setup Step 11

If everything was installed correctly, xpcv starts now. After a short loading period you should see this screen.

Now you can start to create your first computer vision project. Follow the instructions below for further advice.

Raspberry Pi Raspbian Bundle

Installing Raspbian on an SD card

The provided XPCV application bundle was generated for the latest release of the Raspbian linux distribution. If you do not have already an SD card with a recent Raspbian version on it, you can use the Raspberry Pi NOOBS Setup Guide to install Raspbian. After installing, upgrade the installed packages using the following command:
sudo apt update && sudo apt upgrade

Activating the Raspberry Pi Camera

The Raspberry Pi camera is disabled by default. To activate it, open the Raspberry Pi Software Configuration Tool in a Terminal using the command sudo raspi-config. Navigate to the menu option Enable Camera and hit return to enable support for the Raspberry Pi camera.

Enable Raspberry Pi camera using the raspi-config tool

Enable Raspberry Pi camera using the raspi-config tool

After activating the camera, close the tool and reboot the system for the changes to take effect.

Download

From the console you can download xpcv using the wget command. The --content-disposition in necessary to ensure that wget reads the proper file name.

wget --content-disposition https://www.xpcv.de/download/raspbian-jessie-bundle/?wpdmdl=388

Unpack and Install

The xpcv bundle for Raspbian comes as a compressed archive file. You can extract it using the terminal application tar. First open a terminal and navigate to the directory which contains the xpcv-1.0-beta-raspbian. To extract the archive, you can use the following command:

tar -xvf xpcv-1.0-beta-raspbian.tar.gz

After extracting, navigate to the generated xpcv directory (cd xpcv). There you can find a script for installing some packages on which XPCV depends. To install these packages, simply execute the script file:

./install_dependencies

The script uses sudo to acquire root privileges. Therefore, enter the password of the current user when you are asked.
If the installation of the packages was successful, you should be able to start XPCV by calling

./start_xpcv

Raspberry Pi SD Image

Writing an SD Card Image under Windows

SD Image Writing under Windows

Please download “Win32DiskImager” from here and install the program.

  1. Load the “Image File” by clicking on the folder icon and browsing for the downloaded *.rpi-sdimg file.
  2. Select the SD card you want to write the image to. Make sure to select the drive letter according to your SD card. Otherwise all data on the device would be corrupted.
  3. Click “Write” to copy the image to your SD card.

Writing an SD Card Image under Linux

Note: When using the “dd” tool you may overwrite a partition of your machine. If you select the wrong drive you could delete your primary Linux partition. Please be careful.

  1. Before inserting the SD card in your computer, run df -h command in order to see which devices are currently mounted.
  2. Then, insert the card into your machine and run df -h again. You will find some additional drives now. It will be listed as “/dev/mmcblk0p1” or “/dev/sda”. In our case it was listed as “/dev/sdb”.
  3. Finally, run the following command to write the image:
    sudo dd bs=4M if=~/dst-image-base-raspberrypi3.rpi-sdimg of=/dev/sdb

First Boot

Boot on Raspberry Pi

Eject the SD card from your PC and insert it into your Raspberry Pi 2/3. Then connect all the peripherals necessary:

  • Ethernet cable
  • Keyboard and mouse (USB)
  • Display (HDMI)
  • Camera (optional, either Raspberry Pi camera module (CSI port) or webcam (USB)
  • Power supply (Micro-USB)

After a few seconds you should see the screen on the left. A few moments later you will be presented the main window.

You can now start configure you first project. Read below to learn how to set up your Raspberry Pi as a Remote Device.


Getting Started

Overview

Basically, xpcv is a software framework for efficient development of machine vision solutions. By using xpcv_gui you can graphically configure algorithms using pre-compiled modules (plugins) and immediately run them in real-time. Furthermore, xpcv is able to run on heterogeneous hardware and software platforms. It includes a seamless communication between a large number of devices in order to realize distributed and connected image processing systems. A growing number of ready-to-use algorithms for a vast amount of application scenarios as well as the support of significant input devices speeds up the development of complex applications.

Key Features

  • Modularity
  • Cross-Platform Development
  • Performance
    Software Development Kit (Coming soon)
  • Graphical User Interface
  • Remote Configuration and Debugging

Introduction

Main Window

When you start xpcv_gui you will initially be presented an empty workspace window and some graphical interface items. The main window is structured as follows.

Main Window

  1. Main Menu:
    This menu unfolds on click and comprises all application and project relevant functionality.
  2. Start Project:
    The project can be run (and stopped) by means of this button.
  3. Editor:
    Once a project has been compiled or loaded successfully, it is displayed within the editor window.
  4. Dashboard:
    All user dashboard elements that are part of a projects are presented in the dashboard window.
  5. View Changer:
    Switches between the current view of editor window and dashboard window:

    • Single View – Only one of both is displayed.
    • Split View – Both are presented in a split window configuration.
    • Multi View – Two separate windows will appear.
  6. Project Tree:
    The structure of the project is presented as a tree view for fast access of modules.
  7. Module Settings:
    This interactive view enables the user to modify and control the behavior of the projects modules.
  8. Message Log:
    In this area all messages are logged. This includes:
    • Informations – Status messages that indicate normal behavior of the running project.
    • Warnings – Status messages that indicate alarming behavior of the running project.
    • Errors – Status messages that indicate critical failures and crashes of the running project.

Project Editor

Computer vision projects are assembled using modules, so called plugins, and the project editor.

Project Editor

  1. Search Filter:
    Type in the name of a certain module into the filter mask to quickly find a plugin in the library.
  2. Basic Command Menu:
    The most crucial commands for compiling a project can be found here for quick access:

    • Add new module – Opens the module store on the left side of the window.
    • Group modules – Multiple modules within the project editor can be packed to one single container module, for easy deployment and a clear structure of the project.
    • Ungroup modules – Undoes the action of the pack command.
    • Release modules from container – Modules within a container module may be chosen by clicking on it and may be moved out of the container by means of this command.
    • Enable grid – For a clear and visible layout of the modules within the editor window a grid may be activated. Manual layouting is now easier since modules dock on grid points automatically
    • Auto layout – Modules may be placed automatically by means of this command.
  3. Project Level:
    This breadcrumb navigation indicates the current level of the project structure.
  4. Zoom Bar:
    This control allows you to zoom in/out complex projects. Alternatively, on multi-touch enabled devices you can use pinch-to-zoom gesture.
  5. Module Library:
    In this list you find all available modules organized in categories. Add plugins to your project by drag-and-drop.

Dashboard Window

All user dashboard elements in the project are visible on the dashboard window.

Dashboard

  1. Dashboard Elements:

    Dashboard elements such as image displays can be organized freely on the dashboard window.

  2. Dashboard Settings:

    Use these settings to change the organization of the dashboard elements. You can choose between free, grid and flow layout options. Furthermore, the names of the elements can be displayed optionally.

  3. Dashboard Lock:

    The layout of the dashboard elements may be locked to prevent them from further movements.

Plugins, Pins and Datatypes

Project Editor - Input/Processing/Output

Plugins are processing modules that contain certain algorithms such as image filters, feature detectors or input/output functionality. Basically, a project can consist of the following module types:

  • Input
  • Processing
  • Output

In the given example an image and a template are opened by the input plugins named “Image” and “Template” respectively. A container plugin called “Template Matching” encapsulates the functionality for finding the template within the image and presenting the result as an image for displaying on the dashboard.

Plugin

Each plugin is visualized in the project editor by a box containing the following information.

  1. Plugin Name
    Each plugin can be identified by its name. This name may be changed using the Module Settings. By default is the plugin type.
  2. Help
    Documentation for each plugin may be accessed by clicking on the question mark.
  3. Delete
    The plugin may be deleted from the project editor window by clicking on this cross.
  4. Output Pin
    Output pins send data to the next plugin.
  5. Symbol
    Each plugin can be identified by a unique symbol.
  6. Input Pin
    Input pins receive data from the previous plugin.

The communication between modules is realized via pins. Output pins are connected to input pins, visualized by blue lines in the project editor. The plugin named “Image” for example contains one single output pin named “Image out”. Basically, we distinguish between:

  • Input Pins – These pins feed the plugin with new data. The pins are located on the left side of a module.
  • Output Pins – These pins output the result of a plugin. The pins are located on the right side.
  • Property Pins – These allow the user to change a module property using pins.

Each pin allows the transfer of information of certain datatypes. If you hover over an output pin, you can see the associated datatype. Some pins accept multiple datatypes.

Output Pin
Input Pin


Example Projects

Template Matching

The user interface of xpcv enables you to quickly assemble, configure and parameterize algorithms via drag-and-drop. Let’s get started by building a simple template matching application.

Template Matching Example Step 1

Begin with the following steps:

  1. Drag a plugin of type ImageFileInput from the plugin library and drop it onto the editor window.
  2. Optionally, change the name of the plugin to “Image” by using the window Module Setting. This window gains you access to all the parameters that the plugin provides.
  3. Click on the property File(s) and select an image or a sequence of images in the appearing file dialog. Observe how the plugin ImageFileInput receives an appropriate output pin.

Repeat this process to add more modules to your project.

Template Matching Example Step 2

Add the following plugins to the editor workspace:

  1. ImageFileInput – This plugin loads the template. Rename to “Template” (optional) and parameterize a file to load.
  2. ImageDisplay – Visualizes the input images on the dashboard. Rename to “Display Image” (optional).
  3. ImageDisplay – Visualizes the template on the dashboard. Rename to “Display Template” (optional).
  4. ImageDisplay – Visualizes the output response function on the dashboard. Rename to “Display Response Function” (optional).
  5. TemplateMatcher – This plugin correlates the input images with the template image.
  6. ColorMap – This plugin translates the output of the TemplateMatcher to an image that can be visualized by the plugin image display.

In order to allow data to be transferred from one plugin to another we have now to connect input and output pins in accordance to the desired functionality. Once the plugins have been connected you can use the Auto Layout function to automatically re-arrange the plugins in the workspace.

Template Matching Example Step 3

Connect the plugins as shown. The plugins are connected by clicking on an output pin, dragging it to an input pin and dropping. Now the application is ready to be run.

Template Matching Example Step 4

Click on Start Project and switch to the dashboard to see the result:

  1. This display shows the input image.
  2. This display shows the template image.
  3. This display shows a color coded response map, i.e. the colors show how good the template matches a certain position in the input image. The blue spot indicates the best possible match.
Template Matching Example Step 5

In order to improve clarity and comprehensibility a set of modules can be bundled to a plugin container using the Group modules function. Mark the two plugins TemplateMatcher and ColorMap either with left-click while holding Ctrl key or by drawing a rectangle around it with the left mouse button hold. Then click on the action Group modules in the Basic Command Menu. Both plugins are packed into a new container plugin.

If you like you can edit this newly created container and give it a meaningful name, in this case “Template Matching”. Double click on the container to have a look inside. Mind how the level menu increases (1). Furthermore, xpcv automatically generates pin objects (2) that connect to the input and output pins of the plugin. By clicking on the level menu you can return to the top level of the project.

Template Matching Example Step 6

Finally, save the project for future purposes by using the Main Menu.

Raspberry Pi as Remote Device

With this example you will learn to set up your Raspberry Pi as a remote device. You can use this functionality to utilize your Pi as an network camera, e.g. to monitor the entrance of your house. In this tutorial, we will set up the Pi as the Remote Server or Host. Your PC will act as a Client which is able to configure the Pi. Furthermore, you will see how easy it is to stream images between host and client device.

Find out your Pi’s IP

First, you have to find out the IP address of your Raspberry Pi. If you know the IP assigned to your Pi you can skip this step. Alternatively, you may use your router configuration to find out the IP.

Raspberry Pi as Remote Device - 01 - Close Application
Raspberry Pi as Remote Device - 02 - ifconfig

Follow the steps below in order to retrieve the IP directly from you Pi:

  1. Close the application using the Main menu
  2. Login: root (no password set)
  3. Use ifconfig to retrieve the IP address.
  4. Now, restart xpcv by starting /usr/bin/xpcv_gui or simply by rebooting you Pi.

Set Up Remote Connection

In this case we use a Windows 10 machine (as client) to remotely configure the Raspberry Pi (as host). Before you proceed, please make sure your Pi is both running and connected to your local network.

Raspberry Pi as Remote Device - 03 - Add RemoteServiceClient

In order to connect to the Raspberry Pi you will need to add the RemoteServiceClient plugin.

  1. Open the module store and drag the plugin to your project.
  2. Under “Module settings” click on the property named “Host” and insert your Pi’s IP.
  3. Now switch to “Actions” (hand icon) and click on “Connect”.

Observe the log window. You should see a notice of successful connection on both your PC and your Raspberry Pi.

Raspberry Pi as Remote Device - 04 - Start Project

  1. Now, you can start your remote project by either clicking on the client’s “Start Project” button or by using the “Start Remote Project” button in the actions menu.
  2. Continue to configure your remote project by double-clicking on the RemoteServiceClient plugin in order to enter the remote project.

Configure Remote Device

Within the RemoteServiceClient plugin you find the project of the remote device. Here you can add plugins and edit properties or module connections. You can also add connections between your remote host and your local client. Follow the steps below to set up a minimalistic remote project containing a camera that retrieves images and sends them to your local machine.

Raspberry Pi as Remote Device - 05 - Add Camera

  1. Add a CameraInput plugin from the module store to your remote project.
  2. Select a camera that is connected to your Raspberry Pi from the properties menu. Generic USB cameras, e.g. webcams are listed as “OpenCV”.
  3. If there is no camera present, you will see an error in the log window.
Raspberry Pi as Remote Device – 06 – Add Display

  1. Now add an ImageDisplay plugin to your project.
  2. Then connect the output pin of the CameraInput to the input pin of the ImageDisplay.
Raspberry Pi as Remote Device – 07 – Release Display

The display you have added is now part of you remote project, i.e. it will appear on the GUI of your Raspberry Pi. Instead you want to show it on your local machine. Use the simple method below in order to establish a connection that transfers the remote camera output to your local display.

  1. Select the ImageDisplay plugin.
  2. Then, click on the “Release modules from Container” button.

This causes the ImageDisplay plugin to be moved from your remote to your local device. Furthermore, the connection between host and client is established automatically. Within the RemoteServiceClient you will see an outgoing remote pin connected to your CameraInput plugin. Click on “New project” in the Project Level menu in order to switch back to your local machine. Here you will see an ImageDisplay connected to the RemoteServiceClient plugin. If you open the dashboard, e.g. by using split view, you will now see the live video stream from the camera connected to your Raspberry Pi.

Raspberry Pi as Remote Device – 08 – Remote Connection
Raspberry Pi as Remote Device – 09 – Local Connection