Eclipse and Debian

Eclipse is a renown IDE with a large community and many plugins that enable it to be used for multiple frameworks and languages like C++ and PHP, in addition to the historical support for Java.
C++ can be thought of as a high-level abstraction layer on top of C. More importantly it is an object oriented language.

Using an IDE or not is really a personal choice. I personally enjoy coding with auto-completion, having the IDE reminding me of function parameters order and type, generating getters and setters, but also making typing faster and less error-prone, not to mention online error checking and highlighted errors right in the source files.
But using an IDE can also be more a curse than a blessing if you really do not master what's going on under the hood.

At Yakaz, we make some use of C++ in our backends. We like when things are done as they should be, often making no compromise on performance. And this often rhythms with C++ to my ears. (Did you spot any troll in disguise? Sorry for that :-P)

In this post I'll present the process of creating a C extension coded in C++ using Eclipse, then adapt the project in order to package it for Debian.

Our Eclipse project

Let's write a simple library that can act as a memory log. It stores a list of strings in memory and permits to retrieve and clear them.
The primary goal is to make it work the exact same way when used from C and from C++.

I'll be using Eclipse Indigo IDE for C/C++ Linux Developers (includes Incubating components). Chances are you can use you own version and flavor of Eclipse, and install the CDT plugin.

Let's start by creating a new C++ Project: menu File/New/C++ Project.
Then select the Shared Library project type and give it the name libmemlog.
Finally click Finish.

Figure 1: New project creation

Figure 1.1: Project settings

Let's add a bit of structure by creating two folders: src/ and include/. The former will contain the source code of the library, and the latter will contain the necessary header that will typically be installed under /usr/include/.
To do so right-click on the project in the Project Explorer tab on the left, choose New/Folder, give the folder name and hit Finish. Repeat for the second folder.

Figure 1.2: Adding folders to the project

Similarly, create a file named memlog.cpp under the src/ folder, and a file named memlog.h under the include/ folder,
To do so, right click on the project and choose New/Source File and New/Header File respectively. Don't forget to write the file extension in the dialog.

Creating a C shared library in C++

We now have a C++ source file and a C header.
The exposed functions should be accessible in C, so we need not to expose any C++ specific constructs in the header. However we are free to use C++ within the source of the library as only the compiled code will get executed.

The library code

Below are the source files for our library:

libmemlog/include/memlog.h


#ifndef __MEMLOG_H__
#define __MEMLOG_H__

#ifdef __cplusplus
#include <cstddef>
#else
#include <stddef.h>
#endif

#ifdef __cplusplus
extern "C" {
#endif

typedef struct MemLog MemLog; // opaque structure
typedef const char *MemLogEntry;

MemLog* memlog_new(size_t maxEntries);
void memlog_delete(MemLog* log);

void memlog_append(MemLog* log, MemLogEntry entry);
void memlog_clear(MemLog* log);
size_t memlog_size(MemLog* log);
MemLogEntry memlog_get(MemLog* log, size_t index);

#ifdef __cplusplus
} //! extern "C"
#endif

#endif //! __MEMLOG_H__

Note the lack of comments. Don't try this at home, I'm a professional!
More seriously, I won't comment nor talk about this little program as the code should speak for itself, and it's not the main matter.

libmemlog/src/memlog.cpp


#include <memlog.h>

#include <vector>
#include <string>
using namespace std;

// Actual definition
class MemLog {
private:
	const size_t mMaxEntries;
	vector<string> mData;

public:
	MemLog(size_t maxEntries)
	: mMaxEntries(maxEntries)
	{
	}

	virtual ~MemLog()
	{
	}

	void append(string entry)
	{
		if (mMaxEntries > 0 && mData.size() == mMaxEntries)
			mData.erase(mData.begin());
		mData.push_back(entry);
	}

	void clear()
	{
		mData.clear();
	}

	size_t size()
	{
		return mData.size();
	}

	string get(size_t index)
	{
		if (index >= size())
			return "(null)";
		return mData[index];
	}

};



MemLog* memlog_new(size_t maxEntries)
{
	return new MemLog(maxEntries);
}

void memlog_delete(MemLog* log)
{
	if (log != NULL)
		delete(log);
}

void memlog_append(MemLog* log, MemLogEntry entry)
{
	if (log == NULL) return;
	log->append(entry);
}

void memlog_clear(MemLog* log)
{
	if (log == NULL) return;
	log->clear();
}

size_t memlog_size(MemLog* log)
{
	if (log == NULL) return 0;
	return log->size();
}

MemLogEntry memlog_get(MemLog* log, size_t index)
{
	if (log == NULL) return "(null)";
	return log->get(index).c_str();
}

If we were to manually write a Makefile for this project, here is how it would look like. It is basically a rough equivalent for Eclipse generated makefiles.

libmemlog/Makefile


.PHONY: all install clean uninstall

INSTALL = install
UNINSTALL = rm -Rfv

INSTALL_PREFIX := /usr

CXXFLAGS += -O3 -g0 -fPIC -I./include
LDFLAGS += -fPIC -shared -Wl,-export-dynamic

SRC_DIR := src
INCLUDE_DIR := include
OUTPUT_DIR := Release

TARGET := $(OUTPUT_DIR)/libmemlog.so
SRC := $(SRC_DIR)/memlog.cpp
HDR := $(INCLUDE_DIR)/memlog.h
OBJ := $(SRC:%.cpp=%.o)
OBJ := $(addprefix $(OUTPUT_DIR)/, $(OBJ))

all: $(TARGET)

$(OUTPUT_DIR)/:
	mkdir $@

$(OUTPUT_DIR)/$(SRC_DIR): | $(OUTPUT_DIR)/
	mkdir $@

$(TARGET): $(OBJ) | $(OUTPUT_DIR)
	$(CXX) $(LDFLAGS) -o $@ $<

$(OUTPUT_DIR)/%.o: %.cpp | $(OUTPUT_DIR)/$(SRC_DIR)
	$(CXX) -c $(CXXFLAGS) -o $@ $<

install:
	$(INSTALL) -m 644 -t $(INSTALL_PREFIX)/lib/ $(TARGET)
	$(INSTALL) -m 644 -t $(INSTALL_PREFIX)/include/ $(HDR)
	ldconfig

clean:
	rm -Rf $(OBJ) $(TARGET)

uninstall:
	$(UNINSTALL) $(INSTALL_PREFIX)/lib/$(TARGET) $(addprefix $(INSTALL_PREFIX)/include/, $(HDR))
	ldconfig

To buy you some time, here is an archive containing the files:
Download libmemlog.tar.gz

And the alternative Makefile:
Download Makefile

Explanations

We used C++ internally for the simplicity of this higher level language, instead of growing our own array and making copies of const char *. Note that const char * to string conversion is implicit. Using string permits us to have our own copy of the provided strings, and to return a temporary const char * buffer out of our string using the str() member function. The memory management is simplified using C++ idioms.

We expose a simple C style API. One function creates an "object" and returns an opaque pointer, that other functions need as first parameter. The real type (as opposed to partial type) of the opaque pointer is a C++ class, but this information is only known when we compile the library. We did not use the C++ class keyword in memlog.h so that it won't mess with C compilers.

As an additional effort, we detect whether the file is being used in a C or C++ context and adapt the standard include files accordingly. We do one more thing, if C++ is detected, we surround the code with extern "C" { and }. This piece is very important when compiling and linking the extension with the C++ compiler. It tells it to remove a special treatment usually associated with C++ functions called "name mangling". It is a process that takes a function name and its signature and generates a new name, encoding the signature along the original function name. Remember that C++ is built on top of C and the latter does not permit function overloading (same name, different signature) but the former does, using this precise trick. A C client of this API should still be able to link against the right symbols in the library, hence it needs the function names to be unmangled. Note that within an extern "C" section in C++, function overloading is not permitted.

You may have noticed the -fPIC flag in compilation and -fPIC -shared flags in linkage. A shared library will get loaded in some virtual space address by the executable. This address can change from executable to executable and even from invocation to another, hence the need for Position Independent Code (-fPIC).
Likewise for the -Wl-export-dynamic linkage flag. If you want your library to be usable dynamically through functions like dlopen(3), you will have to make all exported symbols present in the dynamic symbol table. -fPIC is also required for such a use.

To install a library on your system, it is required to make it available under /usr/lib or a few other paths. We use the install command for this purpose as well as for setting permission bits. Once the library is in place, we need to run ldconfig(8) so that the loader will be able to resolve the library name to the filename.

Compiling and installing

If you were to use the alternative Makefile

Simple enough, here are the required steps:


make
sudo make install

Note:

The makefile has a an uninstall target to when you're done with this library and desire to remove it, simply do:
sudo make uninstall

Very simple... but the hard work has been done manually in the Makefile. It was not too complicated for such a little project, but in real project you'd better stay in your IDE.

Using Eclipse

To continue consistently using Eclipse to build the project: Select a build configuration. Alternatively, use Project/Build All in the menu, but this will also compile any other currently opened projects in the workspace.

Figure 2: Selecting a build configuration and compiling

Note that you will encounter errors because Eclipse currently does not properly configure the project for the compilation and linkage of a shared library. As explained earlier, we need to compile and link PIC.

Figure 2.1: Compilation errors

Transcript of the image

**** Build of configuration Debug for project libmemlog ****

make all
Building file: ../src/memlog.cpp
Invoking: Cross G++ Compiler
g++ -O0 -g3 -Wall -c -fmessage-length=0 -fPIC -MMD -MP -MF"src/memlog.d" -MT"src/memlog.d" -o "src/memlog.o" "../src/memlog.cpp"
Finished building: ../src/memlog.cpp

Building target: liblibmemlog.so
Invoking: Cross GCC Linker
gcc -shared -o "liblibmemlog.so"  ./src/memlog.o
/usr/bin/ld: ./src/memlog.o: relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
./src/memlog.o: could not read symbols: Bad value
collect2: ld returned 1 exit status
make: *** [liblibmemlog.so] Erreur 1

**** Build Finished ****

Let's fix that by tweaking the project build settings:

Right-click on the project and choose Properties at the very bottom.
Then select C/C++ Build/Settings in the list on the left
Don't forget to select [ All configurations ] in the Configuration drop-down at the top.
Failing to do so will result in a properly configured Debug configuration and misconfigured Release configuration.
Then in the Tool Settings tab, select Cross G++ Compiler/Miscellaneous in the list
Finally, check the Position Independent Code (-fPIC) tick

This is almost it. If you take a look at the gcc command manual, you will read:

OPTIONS

Options for Linking

-shared

Produce a shared object which can then be linked with other objects to form an executable. Not all systems support this option. For predictable results, you must also specify the same set of options that were used to generate code (-fpic, -fPIC, or model suboptions) when you specify this option.[1]

[...]

FOOTNOTES

On some systems, gcc -shared needs to build supplementary stub code for constructors to work. On multi-libbed systems, gcc -shared must select the correct support libraries to link against. Failing to supply the correct flags may lead to subtle defects. Supplying them in cases where they are not necessary is innocuous.

The emphasized part makes us check whether Eclipse now gives the -fPIC option to the linker. It does not.
Select Cross G++ Linker/Miscellaneous in the list and manually enter -fPIC in the Linker flags text zone.

Also, you may have noticed that the output library is named liblibmemlog.so instead of libmemlog.so. This is because we wanted to name the project with the lib prefix for clarity.
Let's remove the superfluous prefix: select the Build Artifact tab, and remove lib in the Output prefix.

Figure 2.2: Fixing the output artifact name

At this stage, I'd recommend to clean the project before trying to rebuild. It seems that Eclipse (or its Makefile) detects automatically that it is needed.

Manually installing

Let's merely reproduce the steps of the Makefile install target:

Locate the compiled library. It's either libmemlog/Debug/libmemlog.so or libmemlog/Release/libmemlog.so
Install the library: cp libmemlog/Release/libmemlog.so /usr/lib/ sudo chmod 644 /usr/lib/libmemlog.so
Install the header: cp libmemlog/include/memlog.h /usr/include/ sudo chmod 644 /usr/include/memlog.h
Tell the system about the new library, for the loader to be able to resolve the "memlog" library to the actual filename: sudo ldconfig

The C and C++ client codes

Now that we have a brand new library, we can't wait using it!

We'll be testing it with a C and a C++ client. But, for brevity and demonstration purpose, only the C client code is exposed in this page. Still, both C and C++ versions are available for download.

Like for the library, create a C (resp. C++) project, name it testmemlogC (use a different name like testmemlogCpp for the C++ one).

Figure 3: Creating the client project

Create a src/ folder here too, it's always cleaner.

testmemlogC/src/testmemlog.c


#include <stdio.h>
#include <stdlib.h>

#include <memlog.h>

int main(int argc, char* argv[])
{
	int i, s;

	if (argc == 1)
	{
		printf("Usage: %s max_entries [entry] ...\n", argv[0]);
		printf("    max_entries   Maximum entries of the log files. Use 0 for unlimited.\n");
		printf("    entry         An entry to write in the memlory log.\n");
		return 1;
	}

	MemLog* log = memlog_new(strtoul(argv[1], NULL, 10));
	printf("Size = %lu\n", memlog_size(log));

	for (i = 2 ; i < argc ; i++)
	{
		printf("Adding \"%s\"\n", argv[i]);
		memlog_append(log, argv[i]);
		printf("Size = %lu\n", memlog_size(log));
	}

	printf("\nActual content:\n");

	for (i = 0, s = memlog_size(log) ; i < s ; i++)
	{
		printf("%d)\t\"%s\"\n", i, memlog_get(log, i));
	}

	memlog_delete(log);

	return 0;
}

Feel free to remove the debugging "Size" and "Adding" printf if you trust what the library is doing internally.

If we were to manually write a Makefile for this project, here is how it would look like. It is basically a rough equivalent for Eclipse generated makefiles.

testmemlogC/Makefile


.PHONY: all clean

CFLAGS += -O0 -g3
LDFLAGS += -lmemlog

SRC_DIR := src
OUTPUT_DIR := Release

TARGET := $(OUTPUT_DIR)/testmemlog
SRC := $(SRC_DIR)/testmemlog.c
OBJ := $(SRC:%.c=%.o)
OBJ := $(addprefix $(OUTPUT_DIR)/, $(OBJ))

all: $(TARGET)

$(OUTPUT_DIR)/:
	mkdir $@

$(OUTPUT_DIR)/$(SRC_DIR): | $(OUTPUT_DIR)
	mkdir $@

$(TARGET): $(OBJ) | $(OUTPUT_DIR)
	$(CC) $(LDFLAGS) -o $@ $<

$(OUTPUT_DIR)/%.o: %.c | $(OUTPUT_DIR)/$(SRC_DIR)
	$(CC) -c $(CXXFLAGS) -o $@ $<

clean:
	rm -Rf $(TARGET)

To buy you some time, here is an archive containing the files:
Download testmemlogC.tar.gz
And the alternative Makefile:
Download Makefile

And here is its C++ twin:
Download testmemlogCpp.tar.gz
And the alternative Makefile:
Download Makefile

Explanations

We simply call the C functions from a C program. This will work as expected, even if the library in itself is C++ compiled code. C++ classes will get created, vector will get used, even new and delete will be run, but as the library relies in turn on libstdc++.so, everything will simply work from a C context!

We need to link against our library so that the linker will know what to do with calls to memlog_* functions. The include in themselves only describes how to call the functions: ie. their arguments types, their return types, and their names. It does not tell what to execute when calling the function.

Using a shared library permits to change the implementation without breaking the clients most of the time. Obviously deleting or changing a function signature or behavior will break them.
If you need to do more complicated things, you will need to use "symbol versioning" using the -Wl,-version-script=Symbol.map linkage flag. That's beyond the scope of this article.

Compiling and executing

If you were to use the alternative Makefile

Simple enough too, here s the required step:


make

Handmade Makefiles can get quite complicated for real world projects, so let's stay in Eclipse and see how it works.

Using Eclipse

Guess what? It won't compile either! But this time Eclipse is not the one to blame. Don't worry, I'll take the responsibility on myself...

Figure 4: Compilation errors

Transcript of the image

**** Build of configuration Debug for project testmemlogC ****

make all
Building file: ../src/testmemlog.c
Invoking: Cross GCC Compiler
gcc -O0 -g3 -Wall -c -fmessage-length=0 -MMD -MP -MF"src/testmemlog.d" -MT"src/testmemlog.d" -o "src/testmemlog.o" "../src/testmemlog.c"
Finished building: ../src/testmemlog.c

Building target: testmemlogC
Invoking: Cross GCC Linker
gcc  -o "testmemlogC"  ./src/testmemlog.o
./src/testmemlog.o: In function `main':
/data/Article/BlogTech/EclipseCppCExtension/workspace/testmemlogC/Debug/../src/testmemlog.c:18: undefined reference to `memlog_new'
/data/Article/BlogTech/EclipseCppCExtension/workspace/testmemlogC/Debug/../src/testmemlog.c:19: undefined reference to `memlog_size'
/data/Article/BlogTech/EclipseCppCExtension/workspace/testmemlogC/Debug/../src/testmemlog.c:24: undefined reference to `memlog_append'
/data/Article/BlogTech/EclipseCppCExtension/workspace/testmemlogC/Debug/../src/testmemlog.c:25: undefined reference to `memlog_size'
/data/Article/BlogTech/EclipseCppCExtension/workspace/testmemlogC/Debug/../src/testmemlog.c:30: undefined reference to `memlog_size'
/data/Article/BlogTech/EclipseCppCExtension/workspace/testmemlogC/Debug/../src/testmemlog.c:32: undefined reference to `memlog_get'
/data/Article/BlogTech/EclipseCppCExtension/workspace/testmemlogC/Debug/../src/testmemlog.c:35: undefined reference to `memlog_delete'
collect2: ld returned 1 exit status
make: *** [testmemlogC] Erreur 1

**** Build Finished ****

Yep! you've found the error we've made. Eclipse cannot yet read minds (though I'm working real hard on this plugin...) so it didn't find out that you were missing a link time library.
To tell Eclipse, open the project properties, select C/C++ Build/Settings in the left list, then in the Tool Settings tab select Cross GCC Linker/Libraries, and add an entry in the Libraries list.

Figure 4.1: Linking to the library

Testing

The first argument is the maximum backlog, use 0 for unlimited. The following arguments are each log entry to be inserted. If you specify 3 as a limit and provide 5 log entries, only the last 3 will be kept.


./Release/testmemlog
# [Prints the usage message]
./Release/testmemlog 3 One Two
# 0) One
# 1) Two
./Release/testmemlog 3 One Two Three Four
# 0) Two
# 1) Three
# 2) Four

Using Eclipse, the easier is still to open a console and cd into the right directory.

Packaging for Debian

Debian packages are meant to simplify the life of users and therefore of distributors (quite often the programmers themselves). The .deb files basically are archives containing a 2 other archives: one with a few control files in a specific format, and the other with data files to be copied in the file system.
Even if you consider the files and commands required to create a Debian package as completely alien, they really are not complicated and many times very few customization are needed on top of a standard skeleton.

First, I have to point you to some documentation, namely:

The Debian Policy: Presents the structure of Debian packages, but does not help to create them.
Official description: This manual describes the policy requirements for the Debian distribution. This includes the structure and contents of the Debian archive and several design issues of the operating system, as well as technical requirements that each package must satisfy to be included in the distribution.
Debian New Maintainers' Guide: This is where is discussed the real matters. This document describes all the required steps and files to create a package. Optional files and steps are also discussed, making it a complete guide, although nothing replaces experience.
Debian Developer's Reference: Especially chapter 6 and chapter 5 for managing packages.
This document is likely to be interesting to you only if you want to become a Debian contributor, or improve your mastery in the fine art of packaging.

The debian/ folder

Only 4 files are required in this folder: control, rules, changelog and copyright, by order of importance. I describe here their role and content for packaging our library project.

If you want to generate a skeleton: install the dh-make package, name your project folder packagename-version, cd into it and run dh_make --library --createorig (add --copyright gpl|lgpl|apache|bsd to choose a precise license, see the dh_make(8) manpage for more).
This will generate many additional .ex (stands for example) files.

debian/control

This file defines the source package and the binary packages.
A source package is a collection of a .orig.tar.gz, an eventual .debian.tar.gz, and a .dsc file. The first archive is the project source, the second is a separate archive for the debian/ directory, and the third describes and signs the previous files.
A binary package is what permits apt-get and dpkg to install the package in the computer. It consists of the control file, compressed, and an archive of data files to be copied. The source package simply is the source files that permit reconstructing the packages.

debian/control


Source: libmemlog
Section: libs
Priority: optional
Maintainer: Olivier Favre <olivier@yakaz.com>
Build-Depends: debhelper (>= 7.0.50)
Standards-Version: 3.8.4

Package: libmemlog
Section: libs
Architecture: any
Depends: ${shlibs:Depends}, ${misc:Depends}
Description: A simple memory log C library
 libmemlog is a simple C library that helps managing an ordered
 possibly limited list of entries.

Package: libmemlog-dbg
Architecture: any
Section: debug
Priority: extra
Depends: ${shlibs:Depends}, ${misc:Depends}, libmemlog (= ${binary:Version})
Description: A simple memory log C library - debugging symbols
 libmemlog is a simple C library that helps managing an ordered
 possibly limited list of entries.
 .
 This package provides debugging symbols for it.

Package: libmemlog-dev
Section: libdevel
Architecture: any
Depends: ${shlibs:Depends}, ${misc:Depends}, libmemlog (= ${binary:Version})
Description: A simple memory log C library - development files
 libmemlog is a simple C library that helps managing an ordered
 possibly limited list of entries.
 .
 This package provides the associated development files.

The first bloc describes the source package. Note its name is unrelated with the binary packages names.

The second one describes our first binary package: the library itself. It is located under the libs section. The third line specifies that this package need to be compiled for each target architecture. One can use all if there is no architecture dependency, like for scripts or text files. The Description field it composed of a short description on the same line, and a long one, running on the following lines starting with a space. As blank lines separate packages, we have to put a dot "." for a blank line in the description.

The third bloc defines the debugging package, containing only the debug symbols of the library shipped in the previous package. The process of severing the debugging symbols from an object (static or shared library, executable or compilation object) is called "stripping". It is controlled in the debian/rules file.

The fourth package will ship the headers for our library. The first binary package containing the compiled library is necessary and sufficient for depending executables to run properly, but in order to compile an executable using the library, the compiler needs to know the existence and signature of the exported functions, in addition to linking to the compiled library.
We will also use this binary package to ship a debugging version of the library. Assertions and optional runtime validation can be activated in that version. We will only compile it without any optimization and with the maximum debugging information, for ease of debugging.
Because we ship a compiled library in this package, we must set its Architecture to any. If we only shipped our header, all would be sufficient.

debian/rules

This file actually is a Makefile. It controls the majority (if not all) of the package creation process. In its simplest version it delegates every target (%) to debhelper (dh $@). This works especially well for projects using autotools. Unfortunately we have some work to do in order to adapt our Eclipse project to debian packaging.

The file has multiple targets. We will need to override the build and clean ones of course, to delegate to the Eclipse generated makefiles. We will also override the strip step, to add a few arguments.

debian/rules


#!/usr/bin/make -f
# -*- makefile -*-

# Uncomment this to turn on verbose mode.
#DH_VERBOSE=1           # local variable, will only affect dh* commands.
#export DH_VERBOSE=1    # environment variable, will recursively affect all dh* subcommands.
#export DH_OPTIONS=-v

build: build-stamp

build-stamp:
	@# debhelper sanity checks
	dh_testdir
	@# Export build flags in a special file,
	@# that Eclipse includes in its Makefiles.
	dpkg-buildflags --export=make > makefile.defs
	@# Delegate build to Eclipse generated makefiles.
	$(MAKE) -C Release/
	$(MAKE) -C Debug/
	@# Update the target
	touch $@

clean:
	@# debhelper sanity checks
	dh_testdir
	dh_testroot
	@# Remove files created by this Makefile
	rm -f build-stamp makefile.defs
	@# Clean Eclipse subfolders by delegating to its generated Makefiles
	[ ! -f Release/makefile ] || $(MAKE) -C Release/ clean
	[ ! -f Debug/makefile ] || $(MAKE) -C Debug/ clean
	@# Let debhelper to its own clean tasks
	dh_clean

override_dh_strip:
	@# Will strip libmemlog/usr/lib/libmemlog.so and put its debugging symbols in the libmemlog-dbg package.
	@# Won't strip libmemlog-dev/usr/lib/libmemlog_g.so because of the added exclude filter.
	@# (static libraries *_g.a files don't get stripped, but there's no such rule for *_g.so files)
	dh_strip --exclude='_g.so' --dbg-package=libmemlog-dbg

# Delegate any other unknown target to debhelper
%:
	dh $@

.PHONY: build clean override_dh_strip

See the subsection "Required adaptations of our project" for a description of the line 16.

Almost everything has already been explained. We must use make -C Release/ to cd to the Release/ directory and run make inside, as it is how the Eclipse generated makefiles expect to be used.
Note that we test the existence of the makefiles in the clean target. That way it's easier to reuse this Makefile. But more importantly, note that these lines must succeed if no file is present. Writing [ -f Release/makefile ] && $(MAKE) -C Release/ clean would make the clean target fail if the file is absent because logically false AND unknown evaluates to false, and any command that returns an error message (eg.: this test yielding false) is a fatal error for a makefile. Note we could have use the dash prefix "-" to make it ignore the error, but this would print ugly warnings.
Anyway, you should always reread, adapt and test the things you copy-paste!

debian/changelog

Don't be tempted to disregard this file as it is where you control the versions of your package.

It's format is quite simple, but may can be a bit tricky. The first line is composed of the source package name, then its version surrounded in parenthesis, then its target repository with a trailing semi-colon ";", then the urgency keyword followed by an equal sign and a special value.
Then are listed changes details, each entry starting with 2 spaces, an asterisk "*" and a space, and continuation lines starting with 4 spaces.
Finally the maintainer signature, starting with a space, 2 dashes and a space, followed by the maintainer name, a space, its email surrounded in angle brackets "<" and ">", then two spaces and the date in the RFC 2822 format.

Think it's hard? Be reassured, you'll mostly use copy-paste to define new versions. And adding new versions is as easy as adding a new block at the beginning of the file, followed by a blank line. The date -R command can be used to generate a valid date (but don't forget the 2 spaces before it!).
Please, don't be tempted to invent new values, read the manual section instead: here and here for more information.

debian/changelog


libmemlog (1.0.0-1) experimental; urgency=low

  * Initial Release

 -- Olivier Favre <olivier@yakaz.com>  Thu, 08 Dec 2011 11:03:52 +0100

debian/copyright

And finally the least important of the required files (to my eyes at least...), the copyright file. It can be either a freeform file, or start with the Format-Specification: [some URL] line.

I guess the format is not too complicated to infer from the following example, but recall the rules for the debian/control Description field as they apply to the license body.

In the example we used the most flexible license I know of: the 2-clause BSD license. In some countries it is not possible to author a creation and explicitly dropping your rights, making it in the public domain, I use this license to simulate it.

debian/copyright


Format-Specification: http://svn.debian.org/wsvn/dep/web/deps/dep5.mdwn?rev=135
Upstream-Name: libmemlog
Upstream-Maintainer: Olivier Favre <olivier@yakaz.com>
Upstream-Author: Olivier Favre <olivier@yakaz.com>

Files: *
Copyright: 2011, Yakaz
License: BSD-2-Clause

Files: debian/*
Copyright: 2011, Yakaz
License: BSD-2-Clause

License: BSD-2-Clause
 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions
 are met:
 1. Redistributions of source code must retain the above copyright
    notice, this list of conditions and the following disclaimer.
 2. Redistributions in binary form must reproduce the above copyright
    notice, this list of conditions and the following disclaimer in the
    documentation and/or other materials provided with the distribution.
 .
 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
 ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
 A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
 HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
 LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

We'll add some file to the debian/ folder. Some may really be optional, but we will need at the very least the debian/*.install files.

debian/compat

This file merely tells that we need at least the version 7 of debhelper(7).

debian/compat

debian/source/format

Multiple package type exist, the most simple one is native, and the most used one is probably quilt. The latter permits applying different patches while keeping a vanilla version of the source code.

debian/source/format


3.0 (native)

debian/source/options

Here, we control some options for the creation of the source package. If you're using a versioning system, and/or want to build the packages from the project working directory, be particularly picky at what should and what should not be included in the source package. If need be, open the created .deb and .tar.gz files to ensure that no undesired file have landed there.
I personally use the following command inside a script for this purpose, using the git VCS:


if [ -d .git ] && [ "$(git status --porcelain | grep -E '^\?\?' | wc -l)" -gt 0 ]; then
        echo "Clean your repository before!"
        exit 1
if

dpkg-buildpackage -sa -us -uc -tc

Always test on a fresh, clean machine, by checking out your sources and building your package. Some of the advices in this article come from such tests.

debian/source/options


tar-ignore = .git
tar-ignore = .gitignore
# Reproduce any pattern of .gitignore here
# if you want to be able to build a clean package
# from your project's working directory.

debian/libmemlog.install

The debian/*.install files are named after the binary package they represent. Each line represents one or more files to install.
A line is composed of two fields, separated by a space. The first part is the origin filename, relative to the project directory. And the second one is the destination, relative to / (so it is a relative path). The second can be omitted, in which case it takes the same value as the first field (hence the use of a relative path).

debian/libmemlog.install


Release/libmemlog.so usr/lib/

Remember the "Manually installing" section? It's basically what we are doing in this file. Here we only install the compiled library. The permissions are automatically fixed by a designated step in the package creation process.

debian/libmemlog-dev.install

In the development package we have to install the header file, as well as the debugging library.

debian/libmemlog-dev.install


Debug/libmemlog_g.so usr/lib/
include/* usr/include/

No debian/libmemlog-dbg.install file needed!

The dh_strip command automatically knows where to install the files it generates for our debug package, so there is no need to explicitly tell it here.

Remove any Makefile file in the project directory, or debhelper will use it as an autotool makefile to generate the install instructions.

Required adaptations of our project

When building a package, you may notice a few lines on the top:


CFLAGS=-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security
CPPFLAGS=-D_FORTIFY_SOURCE=2
CXXFLAGS=-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security
FFLAGS=-g -O2
LDFLAGS=-Wl,-z,relro

Note:

These values change from distribution to distribution, and the previous are for Debian Sid.
For Debian Squeeze you would see:
CFLAGS=-g -O2 CPPFLAGS= CXXFLAGS=-g -O2 FFLAGS=-g -O2 LDFLAGS=
Note that using the previous flags on another machine may result in strange behavior! I actually had a problem with an executable compiled in that way, and it was some used external library that was complaining...

We will make Eclipse use those flags. Go to the project properties, in C/C++ Build/Settings, and in the Tool Settings tab, choose in turn the 3 developed top items and modify the Command line pattern as follows, adding the part in bold:

Cross GCC Compiler: ${COMMAND} $(CFLAGS) $(CPPFLAGS) ${FLAGS} ${OUTPUT_FLAG} ${OUTPUT_PREFIX}${OUTPUT} ${INPUTS}
Cross G++ Compiler: ${COMMAND} $(CXXFLAGS) ${FLAGS} ${OUTPUT_FLAG} ${OUTPUT_PREFIX}${OUTPUT} ${INPUTS}
Cross G++ Linker: ${COMMAND} $(LDFLAGS) ${FLAGS} ${OUTPUT_FLAG} ${OUTPUT_PREFIX}${OUTPUT} ${INPUTS}

Note that we used parenthesis instead of curly braces. We use a Makefile variable, not some Eclipse variable.

Figure 5: Adding standard build flags in Eclipse makefiles

Unfortunately, Eclipse generated makefiles don't inherit those variables from the environment, so one more step is required: making its makefiles aware of them.
Looking at the makefiles, we see that they import ../makefile.defs. So simply by making such file available, Eclipse makefiles can be tweaked.
This is exactly what the line 16 of debian/rules does.

You may have noticed that we referenced Debug/libmemlog_g.so in debian/libmemlog-dev.install. We need to change the created artifact name for the Debug configuration only accordingly: Go to the project properties, in C/C++ Build/Settings, make sure the active configuration is Debug and in the Build Artifact tab append _g to the Artfact name.

Figure 5.1: Changing the debug library name

All of this seems quite complicated and hidden, I think it is complicated only because Eclipse keeps them hidden. Unfortunately, this is not the only drawbacks of Eclipse.

Important traps and pitfalls!

Whenever you make a change in the project, you should consider if everything is still in place for the Debian packaging.

If the makefiles may be affected, select the Debug/ and Release/ folders in the Project Explorer, right-click and select Delete.
Then right-click on the project and select Build configurations/Build all, in order to force the creation of fresh makefiles.
If you use a VCS, do not simply ignore those folders, be more precise:
- Prefer not to ignore .cproject
- Prefer not to ignore .project
- Ignore Debug/*.d
- Ignore Debug/*/*.d
- Ignore Debug/*/*/*.d
- etc.
- Ignore Debug/artifactName
- Do not ignore Debug/*.mk
- Do not ignore Debug/*/*.mk
- Do not ignore Debug/*/*/*.mk
- etc.
- Ignore Release/*.o
- Ignore Release/*/*.o
- Ignore Release/*/*/*.o
- etc.
- Ignore Release/artifactName
- Do not ignore Release/*.mk
- Do not ignore Release/*/*.mk
- Do not ignore Release/*/*/*.mk
- etc.
I didn't find really more generic rules because folder/**/*.o don't recurse as expected, at least for git. If using git, do not ignore "Debug" or "Release" (without trailing slash), or the whole folder will be completely ignored!
Failing to do so will make a project repository without any makefiles nor ways to recreate some!
If you added a prebuild step (with the Build Steps tab in our most favorite panel of the project properties) that generates source files, you may either (I'm not sure what variant works best):
- In Eclipse, build multiple times, eventually refreshing the makefiles to make them acknowledge the new files AND adapt debian/rules to make a two step compilation by calling $(MAKE) Release/ pre-build and $(MAKE) Release/ main-build separately.
- Create the ./makefile.targets and fill some variables to make them acknowledged:
  CPP_SRCS += ../src/generatedfile.cpp OBJS += ./src/generatedfile.o CPP_DEPS += ./src/generatedfile.d
  If you also create a new folder, take a look at Release/src/subdir.mk and try your best! Keep in mind that such file is generated and depends on compilation flags and other stuff we've configured.
- Or try also with ./makefile.init, included before in the makefile.
Always check that you no absolute path are hidden somewhere, especially in Eclipse generated makefiles. It will be the case if you use any variable like ${workspace_loc:/${ProjName}}.
This should be easy to check, just grep for your project folder absolute path or the parent folder name!

Building the package

You've waited long enough, now let's build our package!

You will need a few packages for that:

build-essential in order to compile anything
debhelper to get all the required tools for our current task

Get into the project directory, containing the debian/ folder, and issue the following command:


$ dpkg-buildpackage -sa -us -uc -tc

A lot of output will be printed, but the overall operation should not take too long. Take a look at the output and watch for any warning, or see if it stopped with an error.

Here is a brief description of the arguments:

-sa: Asks dpkg-genchanges to include the source package in the .changes file.
-us: Don't sign the source package. Don't use gpg to sign the .dsc file.
-uc: Don't use gpg to sign the .changes file.
-tc: Clean the tree after successfully creating the packages.
You may want to disable it for further debugging the package creation process.

Right, but you now wonder what this command actually does: (simplified from dpkg-buildpackage(1))

It prepares the build environment by setting various environment variables.
It checks that the build-dependencies and build-conflicts (from debian/control) are satisfied.
This is used to insure that the computer has the required configuration for compiling (more generally generating) the package.
It calls fakeroot debian/rules clean to clean the build-tree.
It calls dpkg-source -b to generate the source package.
It calls debian/rules build followed by fakeroot debian/rules binary.
It calls gpg to sign the .dsc file (unless -us is specified).
It calls dpkg-genchanges to generate a .changes file.
It calls gpg to sign the .changes file (unless -uc is specified).
If -tc is specified, it will call fakeroot debian/rules clean again.
Finally it calls dpkg-source --after-build.

Well, I guess we've put everything together and built the packages, congratulations! You should see the following files in the parent folder:

libmemlog_1.0.0-1_amd64.changes
libmemlog_1.0.0-1.dsc
libmemlog_1.0.0-1.tar.gz
libmemlog_1.0.0-1_amd64.deb
libmemlog-dbg_1.0.0-1_amd64.deb
libmemlog-dev_1.0.0-1_amd64.deb

I won't give you those file for download for the architecture and distribution reasons exposed earlier, you'd better create them yourself using the described steps.
But I will give you the updated Eclipse project, along with its debian/ folder:
Download libmemlog.tar.bz2

Packaging the client project

Let's start with the C++ client project, here are the main files under the debian/ folder:

debian/control

debian/control


Source: testmemlog-c++
Section: utils
Priority: optional
Maintainer: Olivier Favre <olivier@yakaz.com>
Build-Depends: debhelper (>= 7.0.50),
               libmemlog-dev (>= 1.0.0), libmemlog-dev(<< 1.0.0.)
Standards-Version: 3.8.4

Package: testmemlog-c++
Section: utils
Architecture: any
Depends: ${shlibs:Depends}, ${misc:Depends},
         libmemlog (>= 1.0.0), libmemlog (<< 1.0.0.)
Description: A simple testing utility for the memory log C library
 libmemlog is a simple C library that helps managing an ordered
 possibly limited list of entries.
 .
 This program is a sample executable demonstrating its use.

Package: testmemlog-c++-dbg
Section: debug
Architecture: any
Priority: extra
Depends: ${shlibs:Depends}, ${misc:Depends}, testmemlog-c++ (= ${binary:Version})
Description: A simple testing utility for the memory log C library - debugging symbols
 libmemlog is a simple C library that helps managing an ordered
 possibly limited list of entries.
 .
 This package provides debugging symbols for its demonstration sample executable.

In the source package, we use Build-Depends on libmemlog-dev in order to make explicit our compile time dependency, and in the binary package we use Depends on libmemlog in order to make the latter package automatically installed when installing the former.

For demonstration purpose, I added an explicit dependency on a specific version, with a very tight range. The use of the final dot is quite weird, but it's the best alternative I've found to (<< 1.0.1), which would let versions as 1.0.0.a pass the test. See the description of the Version field of the debian/control file for more information. Test your filters using the following command:


dpkg --compare-versions 1.0.0.a "<<" 1.0.0. && echo true || echo false

Note that the "<" and ">" are deprecated operators as they are in fact aliases for "<=" and ">=", which is at least a bit counter intuitive!

debian/rules

debian/rules


#!/usr/bin/make -f
# -*- makefile -*-

# Uncomment this to turn on verbose mode.
#DH_VERBOSE=1           # local variable, will only affect dh* commands.
#export DH_VERBOSE=1    # environment variable, will recursively affect all dh* subcommands.
#export DH_OPTIONS=-v

build: build-stamp

build-stamp:
	@# debhelper sanity checks
	dh_testdir
	@# Export build flags in a special file,
	@# that Eclipse includes in its Makefiles.
	dpkg-buildflags --export=make > makefile.defs
	@# Delegate build to Eclipse generated makefiles.
	$(MAKE) -C Release/
	@# Update the target
	touch $@

clean:
	@# debhelper sanity checks
	dh_testdir
	dh_testroot
	@# Remove files created by this Makefile
	rm -f build-stamp makefile.defs
	@# Clean Eclipse subfolders by delegating to its generated Makefiles
	[ ! -f Release/makefile ] || $(MAKE) -C Release/ clean
	[ ! -f Debug/makefile ] || $(MAKE) -C Debug/ clean
	@# Let debhelper to its own clean tasks
	dh_clean

override_dh_strip:
	@# Will strip testmemlog-c++/usr/bin/testmemlog and put its debugging symbols in the testmemlog-c++-dbg package.
	dh_strip --dbg-package=testmemlog-c++-dbg

# Delegate any other unknown target to debhelper
%:
	dh $@

.PHONY: build clean override_dh_strip

Nothing particular here. I've removed the build of the Debug target which is not used, and the debugging package name has been adapted.

debian/changelog

debian/changelog


testmemlog-c++ (1.0.0-1) experimental; urgency=low

  * Initial Release
  * Artificial explicit dependency on libmemlog 1.0.0

 -- Olivier Favre <olivier@yakaz.com>  Thu, 08 Dec 2011 17:51:26 +0100

debian/testmemlog-c++.install

debian/testmemlog-c++.install


Release/testmemlog-c++ usr/bin/

Note that the artifact name has been adapted accordingly in the project.

After building the packages with dpkg-buildpackage -sa -us -uc -tc, you should see the following files in the parent folder:

testmemlog-c++_1.0.0-1_amd64.changes
testmemlog-c++_1.0.0-1_amd64.deb
testmemlog-c++_1.0.0-1.dsc
testmemlog-c++_1.0.0-1.tar.gz
testmemlog-c++-dbg_1.0.0-1_amd64.deb

Again, I won't give you those file for download, you'd better create them yourself using the described steps.
Here is the updated Eclipse project, along with its debian/ folder:
Download testmemlog-c++.tar.bz2

And here is the version for the C project:
Download testmemlog-c.tar.bz2

Conclusion

We have used one of the most famous IDE, and created a C++ library that is fully retrocompatible with the C language, and two testing clients: one in C, the other in C++.

We then created Debian packages for each of them. Adapting the automated IDE workflow to packaging has proved to be rather accessible, but also quite error prone.
At least, we saved quite some time and mental healthiness by staying away of the autotools madness!

That said, they are better alternative for cross-platform project compilation, and I hope that those integrate better with packaging systems.
Anybody have experience with CMake, Scons, premake, Waf et al.?

Yakaz Engineering Blog

Yakaz engineers share their tech knowledge and experience.

Using Eclipse to write a C library in C++ and package it for Debian

Our Eclipse project

Creating a C shared library in C++

The library code

Explanations

Compiling and installing

If you were to use the alternative Makefile

Using Eclipse

Manually installing

The C and C++ client codes

Explanations

Compiling and executing

If you were to use the alternative Makefile

Using Eclipse

Testing

Packaging for Debian

The debian/ folder

Required adaptations of our project

Important traps and pitfalls!

Building the package

Packaging the client project

Conclusion

TrackBack

Comments

Search

Recent Posts

Categories

Archives