Pi in the Sky

Understanding eMule (part 1)

Posted on June 28th, 2007 at 3:44pm by Pi.
Categories: Internet.

In my experience, the people around me in general doesn’t seem to understand very well how does work all this of P2P, BitTorrent, eMule/eDonkey, etc. Personally I use eMule, after having tried many other P2P methods.

This article is the first of a series which tries to show how to use eMule efficiently and safely, taking advantage of its particularities and trying to overcome its limitations.

You’re reading the first part of the series Understanding eMule. You can also read the second part, which deals with how to choose the right eMule version and how to configure it, and the third and last part, which talks about how use eMule efficiently with some tricks and knowledge.

P2P networks, and the eDonkey network

First, a bit of background: P2P means peer to peer. It’s a generic method to share files which is based in the users of a network, without needing central servers which store the files. One of the first and more famous examples is Napster, closed long ago. P2P networks are also called file sharing network, and file sharing is a common moniker for this concept. There are a lot of different P2P, based in protocols with a variety of characteristics. To access these networks, a client is needed; a program which understands the protocol of one or more P2P networks, and that is able to exchange data with other clients in those networks.

One of these P2P networks in the eDonkey network (sometimes called the ED2K network). It takes its name from the client program eDonkey2000, created by MetaMachines. Both MetaMachines and eDonkey2000 don’t exist anymore, but there’s a plethora of other clients which use the eDonkey network, which is still very popular.

The characteristics of the eDonkey network are similar to other networks of the same kind: multipart transfer, and indexing servers. This means that it needs servers, and the clients connect to them; they index the files shared by those clients, and provide of search and interconnection services. The servers themselves don’t store or relay files; they only tell to clients which other clients have the files. While a file is being shared by a client, and this client is connected to a server, all the other clients can find and download such file. There’s a bunch of permanent servers, and it’s doubtful that there’s any problem due to lack of servers in the coming years.

This is an important advantage over BitTorrent (I make the comparison, although BitTorrent is not a network really). BitTorrent needs a tracker for each file. It acts in a similar way compared to the eDonkey servers, but for only one file. If the tracker is not available, the file stops being shared, even if there’s someone sharing it in that same moment. In general, torrents don’t last too much, due to trackers. Torrents are more oriented to recent files, and unless the tracker is permanent (quite rare, only for special torrents for permanent distribution), or it’s very popular, more sooner than later the torrent will “expire”. However, in the eDonkey network, it’s enough that only one client is sharing the file to make it available, indefinitely. For me, this is an advantage over BitTorrent, although sometimes I still use BitTorrent for certain things.

Since eDonkey2000 stopped being updated, now the eDonkey network is used mainly by eMule clients. eMule is an open source project which initially aimed at replicating eDonkey2000 features without using ads and being 100% free. Today it’s the most popular eDonkey network client, and in my opinion, the most powerful. The eDonkey2000 clients being used nowadays are heavily outdated. eMule made certain changes to the protocol, and eDonkey2000 made other changes in one of its last versions; as a result, the clients aren’t really compatible between them (as they were Bearshare and Limewire in the Gnutella network some years ago).

In the past, I was of the opinion that eDonkey2000 was the one specifying the protocol standard, since eDonkey was its network. But now the eDonkey2000 clients are outdated, and they’re incompatible with other recent clients. I could give you more reasons, but I’ll simply recomend eMule over other clients.

Furthermore, eMule also uses the Kad network, which is built over the Kademlia file sharing protocol, without needing indexing servers. It works in a similar way to the still popular Gnutella network, in which clients are nodes in a network which only communicate with the nearest nodes; and each node relays to its neighbours the file searches and information about other nodes. En la práctica, la mayoría de los clientes conectados a Kad también están conectados a la red eDonkey, pero aun así es otra manera paralela de buscar y encontrar fuentes.

eMule has other advantages, such as the credit system, which rewards the people who shares. P2P networks are based in the sharing concept; if you don’t share files, you don’t help the network. And as final note, eMule is available in a multitude of languages, including spanish.

eMule limitations; or why does this go so slow?

One of the most important limitations in eMule is also one of the most often forgotten. For you downloading something at 20kb/s, someone else must be uploading it at 20kb/s. For you downloading 2mb, someone else has had uploaded 2mb. There’s no other way! That’s why it’s important that people share.

Unfortunately, in today’s internet, that’s a bit difficult, not only because many times people don’t share things during enough time, but for a technical reason. The internet connections at user level have asymmetric bandwidth. In simple words: the uploading speed is not the same as the downloading speed. For example, I have Shittel cable at three megs, it means I can download up to 3 megabits per second, about 363kb/s for real (if there’s people or servers uploading to me 363kb/s, of course), but I can only upload a bit below 600 kilobits per second, about 73kb/s for real.

Since we can only download at the speed the rest of the world can upload, in a P2P network the speed is determined by the uploading speed, which in general is way less than the downloading speed (and the one they advertise so much). If all the eDonkey network clients had the same connection as I do, the average speed of the network would be 70kb/s, very far from the wanted limit of 360kb/s.

That’s the main reason why the majority of people whine about this or that P2P program, eMule included, doesn’t download at 3 megs, or at 5 megs, or whatever. The other big factor in speed (and complaints) is the number of sources (scorces, in geek beengrish).

Where’s my file?

In popular files, which are shared by many people, it’s easy to have 300 or 500 sources, and probably those files will download at stunning speeds, which in general provoke comments and thoughts of the “why the other files don’t download so fast” kind. As I’ve already told, for you to download 2mb, someone else has to upload 2mb. If a file isn’t very popular, and instead of 300 people, only 3 people are sharing it, bad luck.

In eMule, each client which is sharing a file, or part of a file, is a source. A client asks to another client for a file, and generally the source client puts it in a long queue. Clients usually share more than one file, often hundreds or thousands, and bandwidth is limited. Thus, you enter a queue, and await for your turn. The speed at which you advance in the queue until finally you can download a part of the file depends of the amount of files the source is sharing, the amount of people who is asking for those files, and the file priority assigned by the source.

The only strange thing is the priority thingy. By default, eMule assigns an uploading priority depending of the file popularity. The more popular they are, less priority they have; at the same time, rare files have higher priorities. Let’s imagine that a popular file and another more rare have the same priority. Then, in the queue of the source there are 300 people asking for the popular file, and only three people asking for the rare. Thus, the rare file will be share only once each hundred times, and 99 other times, the popular file which probably has many other sources already. With the priority system, the source gives advantage to the rare file, so the three people asking for it advance in the queue faster. This allows that rare files can be efficiently shared at the same time than popular files.

However, it’s quite usual (and very frustrating) to be downloading something which only has one or two complete sources, and seeing that you’re in position 2963 in the queue. And on top of it, you only keep the queue position if you keep connected; if you’ve advanced up to position 100, and you shut down the computer at night, next day you’re again at the end of that long, long queue. The position in the queue is kept only about 30 minutes after the last petition.

One little detail: you only have one position in another client’s queue. If you ask for another file to that same source client, that’s an A4AF (Asked For Another File). The source will give you (when you get to the appropiate position in its queue) the last file you asked for. eMule manages A4AF files quite well, so it asks first for the file with higher priority, and if it has to ask for another file to the same source, ensures that it’s of even higher priority before doing so.

A small factor to take into account is if the sources are located in USA. For example, the free distribution movie Star Wreck (The Pirkinning) with english subtitles is probable that has many sources in USA, which usually have more bandwidth. These files of “international” scope usually download faster than other files. On the other sides, the last short film of the last semi-famous greek geek director is more than probable that it has very few souces, and only in the peloponesian peninsule, so instead of downloading, it’ll be downdragged over a long and soft slope.

As a final note, except in special circumstances, a source only sends a part of a file (9.28mb) at a time. Once that amount is sent, the asking client is sent to the end of the queue.

The advantages of sharing

eMule uses a method to reward people who shares, called credit system. In eMule, sharing doesn’t mean having one hundred million files shared at the same time, but the total amount of information you upload to other clients. Understanding how this credit system works is beneficious both to oneself and to the network, or in other words, the community of users of the eDonkey network.

Actually, the system is quite simple. Each time a source uploads a part of a file to another client, the client which receives that part keeps the amount of data it received from the source. Depending on the amount of data, it assigns a credit or modifier to that source client. This modifier is applied to the queue position, making the source client to advance in the queue faster, and gets to download positions quicker.

The credit system only works from client to client. In other words, what you upload to a bloke in Jamaica is only valid for that Jamaican bloke’s queue. This also implies that it’s only useful for the files that bloke in Jamaica is sharing. No matter how much porn you’re uploading to that Jamaican bloke (who only has porn in his pornhard disk), it’s not going to be useful to advance in the queue of the bloke from Hong Kong who is sharing that industrial engineering document about the resistance of reinforced concrete under humidity conditions (in PDF format).

I did the example with porn, because I’ve meet people, several people, who affirm that downloading porn makes the rest to download faster. This is only true if the sources you share the porn with are also downloading or sharing the other files you’re interested in. Maybe it was the excuse they made to download porn, saying that this way the other files would go faster ^_^

By the way, you don’t need to have a complete file. Once that you got a part of the file (9.28mb), you start sharing it immediately. Actually, the credit system usually works with the people who is downloading the same file as you do. In large files, for example movies, the credit system makes that you exchange parts with people who has other parts you don’t have yet, and viceversa. The credit system makes the file exchange more agile.

One detail more about the advantages of sharing. There are modifications (called mods) of eMule which allow to not share, something quite selfish. There are people who says that if you don’t upload anything, you download faster, and that when you are uploading, your maximum download speed is reduced. Something like that you can’t upload without stopping downloading or similar nonsense. It’s completely false. It’s true that if you fill up all your uploading bandwidth, the download suffers, due to the collapse of the TCP/IP packet manager. It’s enough to configure eMule to not use all the upload bandwidth. For example, I have 73kb/s uploading bandwidth, but I tell eMule to use only 60kb/s maximum, what leaves a margin to surf the web comfortably.

Sharing is good for the network, and for oneself. If everyone started downloading gigs and gigs, without sharing even a byte, no one would download even a byte. Never.

Ok, but why doesn’t it go faster?

eMule automagically takes care of everything: file priority, both downloading and uploading, searching and finding sources, spreading parts of files, etc. But even this way, knowing something about how all this works helps to fine tune the program and its configuration, understanding why not always things don’t go as fast as we would want, and save some frustrations.

But even if there are perfect conditions, it’s hard to download stuff at maximum speed. The last barrier is imposed by eMule itself, not allowing to download more than you’ve uploaded to a certain ratio. If your maximum upload speed is set to 4kb/s or less, the ration is 3 to 1. Between 4kb/s and 10kb/s, it’s 4 to 1. Beyond 10kb/s, there’s no ratio, so you can download without limit. If you share normally, in proportion to the bandwidth you have, then there’s no problem. The ones with bandwidth less than 4kb/s are modem users, which already have an upload and download bandwidth ratio of 2:1. The ADSL, cable or similar users, usually have an upload and download ratio much higher, i.e. I have about 5:1, but since I have more than 10kb/s, I can download without limit already. This limitation encourages people to share with some more bandwidth. If you have 70kb/s of upload, your computer doesn’t care if you’re uploading 10kb/s or 60, so don’t be mean!

Well, we already know some of the little inside secrets of eMule, and we’re armed with knowledge to exploit fully our 3 megs… But, how? Actually, there’s not a magic spell to exploit your bandwidth to the maximum, always. The eMule limitations are, with little variations, exactly the same as other P2P networks.

However, something can be done to raise the possibilities. First, download many files. In general, everything will go slower, but in average you’ll download faster. With my old bandwidth of 300/128, I was using 100% bandwidth all day if I was downloading 10-15 movies, depending on their popularity. Now, with my 3000/576, the maximum I got was 290kb/s once I was downloading 20 movies. I’m lazy and I don’t care.

Another idea is having a quite sheep taste in music and movies. If you only download the most popular stuff, then everything will go faster.

Conclusion

eMule, and the eDonkey network, the same as any other P2P network, is limited by human and technical reasons. No network is perfect, and you have to find the network and client which adapt better to your needs. My choice is eMule, and with all this huge amount of words I’m trying that all of you know better eMule, so you can use it correctly, in your benefit and in all’s benefit. Knowing and using the tools is much better than calling the computer friend to configure your eMule “so it goes faster” while you prepare to thank him with sexual favours. Or not?

In the next part I’ll talk about more practical issues: some tricks to configure and fine tune eMule, and choosing the right eMule version.

no comments yet.

Understanding eMule (part 2) »« Heboris: Tetris for professionals

Understanding eMule (part 2) »« Heboris: Tetris for professionals

Pi in the Sky is powered by WordPress. Dressed with Vistered Little. Hosted at MochaHost.