[RTEMS BSP] graphic support for raspberry pi

Hi,

After three months development under the guide of other RTEMS developers, the graphic support for raspberry pi bsp is about to complete. Here is a resume of all features added and some remarks for users and developpers.

Mailbox:

As introduced in the previous blog, mailbox is the main way for arm cpu to communicate with videocore gpu. The interface contains two functions for read and write data.

for more information, please checkout this blog.

 

VC:

In order to setup the hardware related properties, the mailbox channel arm->vc is used and many interfaces are available. We offer an entry structure for holding data of request or responce with a function to call gpu. With this support we can retrieve most of hardware status (version, power, temperature …..) and also have access to graphic features (framebuffer allocation, color palette etc…).

for more information, please checkout this blog.

 

CMDLINE:

At the first stage of boot, gpu will read the text file config.txt for a BIOS-like configuration, it will also read the single-line text file cmdline.txt and append the content to the kernel command line which will be passed by ATAG structure to kernel later.

As documented in arm cpu documentation, when the kernel started, register r2 will contain the address of  ATAG structures or the devicetree. For the moment, I’ve used the VC interface to get cmdline instead of parsing ATAG and devicetree. As the boot of VC is the early stage, if we use a bootloader linke UBOOT and append more kernel arguments in cmdline, this may cause problems. So for a long term solution, we will implement a more generic way for arm bsps.

The cmdline is initialization by calling rpi_init_cmdline() at the early boot. The we can get the full cmdline by rpi_cmdline() or a substring started with a given string rpi_cmdline_arg(const char* arg). These functions are decalred in header file <bsp.h>.

 

Framebuffer:

As the cmdline is supported , we are allowed to pass arguments to choose the resolution of framebuffer for display. The command line argument format is

–video=<resX>x<resY>[-<bpp>]

where  numbers <resX>, <resY> and <bpp> are decadic. For the moment I’ve only implemented for bpp=32, (32BIT_X888RGB pixel format) which is the most commonly used, so the <bpp> will be ignored. –video=auto can be used for an auto-detection and auto-configuration.

In order to make use of the full memory on board, we have to configure the physic memory size in file /c/src/lib/libbsp/arm/raspberrypi/startup/linkcmds.

MEMORY {
VECTOR_RAM (AIW) : ORIGIN = 0x0 , LENGTH = 0x8000
GPU_RAM (AIW) : ORIGIN = 256M – 64M, LENGTH = 64M
RAM (AIW) : ORIGIN = 0x00008000, LENGTH = 256M – 64M – 48K
RAM_MMU (AIW) : ORIGIN = 256M – 64M – 16k, LENGTH = 16k
}

Here is a setup for a rip with maximum 256M memory and 64M will be used for GPU (by default). If you are using 512M board, please replace all ”’256M”’ by ”’512M”’ so that the memory can be fully used.

Video character output:

With the support of framebuffer, we can draw anything we want and of course, the characters.

At the early stage of boot, _RPI_initVideo() will be called for the initialization of framebuffer and prepare for drawing ASCII characters on the screen. After the init is done, we can have the access to maxCol and maxRow, which stand for the number of characters in a row and the number of rows in the display.

_RPI_outch(char) can be used to output a character on the screen. The scroll of the screen and the ESC sequences are handled automatically by a state machine.

FB console:

Instead of using serial port, we are now able to redirect all to a framebuffer console (fbcons) and use a graphic text console. To choose the fb console, use cmdline argument –console=fbcons.

Graphic libraries:

Previously, we can build the graphic libraries with the script of project rtems-graphic-tool-kits. In order to make it easier to build add-on graphic packages, I’ve ported them all into rtems-source-builder.

To list all available bsets, you can use a command like:

cd rtems-source-builder/rtems;
../source-builder/sb-set-builder –list-bsets

To build a bset, you can use a command like

cd rtems-source-builder/rtems;
../source-builder/sb-set-builder \
–log=graphic-build-log.txt \
–prefix=$HOME/development/rtems/bsps/4.11 \
–with-rtems-bsp=raspberrypi \
–host=arm-rtems4.11   4.11/graphics/t1lib.bset’

If you are using cross-compiling, no installation will be performed. You
may need –pkg-tar-files to creat tar packages and install them manually.
Since microwindows and nxlib depend on previous libraries, they should be
installed afterward.

More usage please checkout RTEMS wiki.

 

codes are available on my github:

*  [https://github.com/yangqiao/rtems rtems Git Repo]
*  [https://github.com/yangqiao/rtems-source-builder rtems-source-builder Git Repo]
*  [https://github.com/yangqiao/rtems-tools rtems-tools Git Repo]

More details of implementations are documented in RTEMS dev wiki

Advertisements

[RTEMS bsp] raspberry pi graphic text console with frame buffer

Hi,

After the frame buffer and mailbox are implemented, I’ve tried to implement a graphic text console with the frame buffer.

To include the frame buffer device for your kernel image, you may need to add the configuration macro  CONFIGURE_APPLICATION_NEEDS_FRAME_BUFFER_DRIVER.

Idealy, the console would be redirected to fb console when it is enabled by kernel command line “console=fb”. But for the moment, as I haven’t yet finished the atag parser nor dtb parser, it would be possible to change the primary console by editing file console-select.c, comment the two lines:

90: Console_Port_Minor = BSP_CONSOLE_FB;

91: BSPPrintkPort      = BSP_CONSOLE_FB;

Then all output will be redirected to serial instead of fb console.

If you’ve found some message like this in kernel info output:

[#]fb_fix_screeninfo: smem_start : C100000
[#]fb_fix_screeninfo: smem_len : 12C000

and the video is freezed, try to check if the segement of memory from 0xC100000, with length 0x12C000 is covered in BCM2835_FBMEM_BASE with length of BCM2835_FBMEM_SIZE, defined in raspberrypi.h. If not, please update these two value.

fbcons

I’ve got the driver work on a samsung 27″ display with rpi B. Since I can’t test it with different display by my self. I would be glad if you can join me to test the driver and send me feedback, so that I can improve it .

To try out the graphic console driver, please checkout

https://github.com/yangqiao/rtems/tree/framebuffer

commit 016dc556  fix bug when cleaning screen. out of range

I have more progress on the previous problem. Please checkout the latest blog for more information.

[RTEMS bsp] raspberry pi mailbox and framebuffer implementation

Hi,

In order to communicate with GPU videocore, which control the early boot up and video output, I’ve implemented some structures and functions to simplify the work.

First of all, the common mailbox read/write functions which would be available in <bsp/mailbox.h>:

extern unsigned int  raspberrypi_mailbox_read(unsigned int channel);
extern void raspberrypi_mailbox_write(unsigned int channel, unsigned int data);

These two functions are the lower level functions to read/write the data from/to videocore.

extern unsigned int  raspberrypi_mailbox_read(unsigned int channel);
extern void raspberrypi_mailbox_write(unsigned int channel, unsigned int data);

In general, we have two mailboxes, and the mailbox 0 is what we use in most cases. channel 0 for Power management, channel 1 for Framebuffer, channel 2 for Virtual UART, channel 3 for VCHIQ, channel 4 for LEDs, channel 5 for Buttons, channel 6 for Touch screen, channel 8 for Property tags (ARM -> VC), channel 9 for Property tags (VC -> ARM). More details can be found on rpi firmware wiki.

One thing important to note, the address data that we pass to mailbox should be physical address instead of virtual address, which means that if L2 cache is enabled, the physical address= virtual address + 0x40000000, and this shift value should be 0xC0000000 if L2 cache is disabled.

Although not all details of mailbox are published or hacked, but we can already get the usage of framebuffer channel, and property tags (ARM->VC) from the wiki page or elinux.org.

In order to simplify the use of mailbox property tags, I’ve implemented some structures and functions.  To query or setup a property, we just need to create a structure with all necessary entries, pass the structure to the related operation function and then get the result from the structure.To allocate the framebuffer for exemple, all structures and operation functions are defined in <bsp/vc.h>

typedef struct
{
uint32_t align;
uint32_t base;
uint32_t size;
}bcm2835_allocate_buffer_entries;

int
bcm2835_mailbox_allocate_buffer(bcm2835_allocate_buffer_entries* _entries);

To allocate the framebuffer:

  bcm2835_allocate_buffer_entries allocate_buffer_entries;
allocate_buffer_entries.depth = 32;
bcm2835_mailbox_allocate_buffer(&allocate_buffer_entries);

Then the response of videocore will be retrieved and filled in the structure, which means, we will have the pointer to the framebuffer and its size from the item “base” and “size” of the structure allocate_buffer_entries.

For the moment, not all functions of property tags are implemented as not all of them are frequently used This can be completed in the future.

To allocate a framebuffer for the video output, we just need to well setup the display size, buffer size, color depth, pixel order, alpha mode, virtual offset, overscan, cursors etc, over the mailbox property tags interfaces. Since the mailbox framebuffer channel has been reported to be unstable in some cases, so we decide to use the property tags channel instead. Attention, to initialize framebuffer is a bit special, for all the related tags should be queued in the buffer. These tags shouldn’t be proceeded one after another.

Once we get the framebuffer’s pointer, we can write pixel color values to it for video output. From user space, we can also access the device /dev/fb and use ioctl to get the pointer , and play around the screen.

One problem is posed here about the memory table setup. As we have to active the memory pages that we need in the mm_config_table, I can’t figure out how the cpu choose the base address of framebuffer and it seems impossible to allocate the buffer to a given address. I’ve set up the base address and the length which would cover my situation, but I can’t make sure that this would cover all situations with other displays or rpis. So I invite other developpers to update the base value and the length value so that it would cover most cases. If you’ve found some message like this in kernel info output:

[#]fb_fix_screeninfo: smem_start : C100000
[#]fb_fix_screeninfo: smem_len : 12C000

and the video is freezed, try to check if the segement of memory from 0xC100000, with length 0x12C000 is covered in BCM2835_FBMEM_BASE with length of BCM2835_FBMEM_SIZE, defined in raspberrypi.h. When we try to write to the memory which is not activated in mm_config_table, all process would just freeze. If you are encountering this problem, you may modify the value of the two macros and update it. Normally for the same display the base address would always be the same if the configuration is not changed.

My works can be reviewed on github : https://github.com/yangqiao/rtems/tree/framebuffer

Have fun !

[GSOC2015 RTEMS] Framebuffer support and the port of graphic libraries for raspberry pi

Hi guys,

I’m really glad to join this year’s GSOC, with RTEMS, an open source real-time operating system designed for embedded systems.

Raspberry Pi is a series of popular, low-cost credit card-sized single-board computers, based on an ARM cpu, with the intention of promoting the teaching of basic computer science in schools. It would be great for both the project and RTOS education if RTEMS can have a full support on raspberry pi. The goal of the project is to improve the RTEMS board support packages for Raspberry Pi, which would bring us HDMI graphic output (framebuffer driver and graphic consoles) , port the graphic libraries, integrate them into RTEMS source builder and also the microwindows(Nano-X windows), an open source project aimed at bringing the features of modern graphical windowing environments to smaller devices and platforms.

As for the preparation of development environment, I’ve got a raspberry pi B and a raspberry pi B+ for on board tests, an tty serial to usb cable with FTDI chip, also a qemu raspberry pi emulator for tests.

Mentors assigned  for me would be : jmayes, Pavel Pisa, Josh, Alan Cudmore, Lou Woods. The other RTEMS developers would also guide me to conquer the difficulties.

Here’s a screen shot for the hello test on board

hello_demo_rpi_rtems

Well, that’s all for the introduction. I’ll keep you updated and enjoy the summer coding ! XD

See you soon.

[KDEConnect] Report at the end of GSOC and expectations

Icon-kdeconnect

First of all, I would like to express my gratitude to my mentors Mr. Àlex Fiestas and Mr. Albert Vaca,  and kde hackers Mr. Daniel Vrátil, Mr. Sergio Martins, Mr. Kevin Krammer  etc. Also, thanks for Chinese GDG who supported us to hold a meet-up at Beijing, introducing our projects, discussing and making some noise to get more students involved.

 

For a brief resume, I’ve accomplished the basic backend, ui for iOS platform , and plugins : Ping, MPRIS, Photo Sharing , Clipboard Sync, Mouspad, Battery report, which have been already implemented on Android version.

Besides, I’ve also implemented new plugins for the synchronization of calendar events, reminders, and contacts, using Akonadi to manage these resources.


During this GSOC, I’ve learned a lot on iOS development, and the development on kde shows me the art of its structures and design . For an experience-less like me, it is rather complicated but really fascinated . I’ve spent a lot of time on tests , trying to figure out the correct way to manipulate the resources as I wanted, but I’ve always got unexpected bugs or misunderstood of the way Akonadi works. Thanks for mailing list and patient hackers on IRC, I’ve finally managed to comprehend and make it work. That makes me realize the importance of communication. One should have talk with other experienced people even if it’s a stupid question. That would save a lot of  time and let people get more ideas .

Besides, I’ve also encountered many wired problems but finally resolved by stack overflow and google search, such as: the public key generated by iOS api miss some important info at the head, and that would make the public key shorter and unrecognizable for others. The address book apis on iOS are not as well wrapped as on OSX, and it doesn’t provide an UID for every contact,that would make it complicated when trying to identify a contact with UID. There’s no parser for calendars in icalendar format, so I’ve got to use the c library libical for parsing and create the icalendar format string manually (it works not perfect, maybe we can find a better way or to create a completed library for icalendar generator and parser). A two finger tap guesture will always trigger a single tap guesture, which won’t hurt but I’m still thinking of how can we create a better self-defined guesture based on iOS guesture recognizer without triggering the wrong one.


 

I was planning to take the advantage of libimobiledevice to bring the useful tools on UI or maybe create some new tools. But I’ve found that the library is a bit complicated for me. I’ll need more work to I’ve just given a try to wrap up some basic tool such as : get connected device list, get device infos, get realtime syslog etc. As the time for GSOC is limited,  I didn’t managed to make some real progress on it and these features are not so useful for users. That is one of my expectations since no iTunes provided on linux.

I would also like to see that a BlueTooth 4.0 – Low Energy connection would be supported with a new backend. Through BTLE, we would be able to save a lot on battery and we would also be able to get more cool things from iOS since the notifications are only allowed to be retrieved by BTLE devices (These apis are designed only for smart watches or other devices). I was also told that kdeconnect would be disabled under the restriction of some school/ company wifi . So it’s cool if we can support a BT connection.

For the improvements, I’m working with Alexi Pol for a plugin loader which will load only the supported plugins instead of all plugins, which is being tested. The payload transfer should be encrypted and I would love to add more gestures for the touch pad plugin.

As the iOS 8 is coming, we are sure to take advantage of new apis , such as : add kdeconnect into the costume share sheet so that we can share a photo/ url  outside the app.


At the end, I would like to express my gratitude again to everyone who helped me and who are following this GREAT project! This is my first time making contribution to an open source community and I’ve really enjoyed it. I would have to say, It’s a good begin for me and I’ll keep learning and contributing    🙂

New Plugins for kdeconnect

Now, more plugins have been implemented:

1. Battery report: Report the battery level and display it in the desktop battery notification.

2.Mousepad: A touch pad which allows us to move the cursor, left/right/middle/double left clicks , scroll .

It doesn’t have virtual buttons. Just like a touchpad with some “OSX style” gestures. Use one finger long press to open right click menu, use two fingers to scroll. More gestures are expected to implement.

3.Calendar:  It will not synchronize your whole calendar events. Instead, it will synchronize events that will happen in 24h for our daily work. Once connected with your device, calendar events will be synchronized and you can check it out with “kontact”.

Here is a demo.

snapshot1

As we can see, the event created on iOS has been added to kontact. The red line points out the current time.

 

照片

On iOS, we can see that we have got the event created by kontact, including an event which overlap the current timeline. While the event “Passed ” which has ended before the current time has not been sent to desktop.

There is still a lot of cleanup works (ex: build a real iCal parser for iOS. What I’m using is an ugly function just in order to build a prototype quickly). Any way I’m glad to see it works XD.

In the following days, I’ll try to implement the plugins which would synchronize out contacts , notes, to-do lists .. etc.