Group Project?

Yes! :D

I'm currently looking at the breveCreatures thingy, making it evolve in a more symmetrical way; and would love to see its control being replaced by neural networks, like the original, that ran on the Parralel computing Thinking Machine. Currently it's still way above my head though, but I'm beginning to get to grasps w/ it.

And then do a 'distributed' version of it, everyone running one generation, pooling results together on a server that then redistributes the best creatures etc. Try to make it a screensaver for platforms besides OSX.

This sounds like a great idea and I'd like to provide whatever support I can for it.

I also think that a networked breveCreatures would be a great place to start. I'd be interested in having creatures be able to interact with creatures evolved on other machines, like for races or some sort of combat.

As for the screensaver on other platforms: if anybody has experience writing Windows screensavers and can provide a simple template project for me, it should be pretty easy to port breveCreatures to Windows.

- jon

Team work might be fun!!!

But, on my side, I still have a lot to catch up before being reasonably "operational" and I don't have so much time... :-(

Speaking about ambitious projects, I'm almost sure that there are such things, but most of them are done by people that are working in the field. And my guess is that they don't share there ideas for obvious reasons... Let say that, with all due respect, they are bunches of "walkers" governed by a rule called "Publish or perish"... ;-)

That being said, I also wonder if there was work done on more "ambitious" projects and also what are the current proeminent ideas in the field and who is working in the field and who is doing it just for fun. For the record, I'm not related to this field from my professional life, I just recently find out about all this and I'm quite amaze!!!

Nevertheless, to come back to your suggestion, unless you have a team project in mind, my guess is that a competition might be a good start... But instead of having the best walker, looking for the best walker that could circumvent obstacles (terrain and walker shape being fixed) migth be more challenging... But again, my limited knowledge of breve and the field might have me said unrealistic things... :?

Cheers!

If people want to get a quick start on a competition type of project, let me propose CaptureTheFlag. We used this as part of a Artificial Intelligence class and it was a lot of fun. It should be good for people with all levels of experience -- the agent behaviors can be very simple or very complex, and often the simpler, more elegant behaviors will win!

As for a larger collaboration project, I think one thing that will be very helpful in the long-term is producing reusable class files that can be shared among projects, and in the long term, putting these pieces together in different ways. So, for example, if you're making an evolvable robot controller for a project, it can be useful to think about making it as general as possible so that others can reuse it in the future. I'm trying to do this myself: the breve 2.4 release has a GeneticAlgorithm and an evolvable DirectedGraph genome type which are highly reusable.

- jon

the publish or perish thing. im a bit of a newbie becouse i havent even finnished uni but when i write code i want other people to enjoy what ive created. sure if someone stole my idea or my code i probably would get quite miffed but i had thought that if we helped make something are names would be somewhere as jk often puts in mentoins of other people in the demo code.

anyways if we come up with a project that would make millons we can cross that bridge when we come to it?

i like the breve fighter idea. im sure if we could get a server we could do it.
ive thought about this alot so bare with me. we can give the creatures restraints , (to stop stupied growth or plain overpowering). say 200 points. evolve the creatures much like the creatures screensaver but a number of possible factors can take place. say tourque of joints , size , speed ,maybe we could add a varible for toughness or life points or something. but all this will be governed by a point system. sorry if im not making myself clear , basicly its to stop the creature being the best at everything , if hes big and powerfull he'll have to be slow, if hes quick and nimble he'll have to be weak. the fighting could be based on life points get the amount of tourque from a creatures hit and minus this from the life points, so the creature would figure use its own limbs to hit the opponent it good.
creatures could learn to dodge maybe. Also we could have a terrain with a food source, the best creature from everyones computer could be loadid up and they could compeate , for instance if they get enough food then they can replicate themselves or something daft.

sorry ive been useless at explaning myself. I thourght i would chuck a few ideas out and see what people take of them.

also for a compitoin its a bit of a girly idea but i remember seeing somewhere that we can now affectively have a sun in breve. so may we could make adaptive flowers? staic based flower with moving leaves? each round could be given a water supply and the plants size is based on how much water available. they could grow towards the sun or something.

just chucking it out there. still got a month of final project stuff to do.

if you got any ideas chuck them in even if you think there a bit daft (cant be worse then flowers) :)

I REALLY like the capture the flag idea , maybe some more games like that would be usefull , where people just have to change a small peice of code. tick tack toe or something more ambitous , but where the main code is already made and all thats got to be changed is some tactics.
along these lines would it be possible for a server or something to have the main world code with empty agents , then users could upload there brains into it. this would help against cheating and make sure all the rules where the same for everyone.
Noz

I like the "Capture the Flag" competition too (though I'll have a lot of new concept to catch up :? )!!! Should be a good start. Everything is already setup, all we have to do its give it a try, assuming jk may provide the server, if a server is needed.

I agree with NoZ, it would be good to have more competition-field packages like this one that could be running for a long time on a dedicated server and could be used by who-ever-wants-to-give-it-a-try to try to beat the champion. It's like using the "walker" rule on ourself... ;-) and, who knows, it may leads to new strategies/ideas...

Developping new competition-field packages migth, BTW, be a good starting point for team projects (a bunch of us may be working together to create a new comp. packages)...

The fighter idea with a points constrained system, might be a good example. Seems fun! Though, the hard work would be to setup the field.

I'll start looking at the "Capture the Flag" code and see if I could come up with something.

Shall we try to formalize this thing?
It would be nice if there would be a link from breve main page to the ongoing capture the flag competition for example...
Any idea?

the server side shouldn't be such a big problem I guess, depending on what kind of simulation is run, take breveCreatures for instance: the 'makeup' of the creatures is just an XML file, and a typical run (say 10 generations) could easily take half an hour, so even if there were several tens of people running the simulation, the server would just have to maintain some kind of database which you access every 30 minutes, to send your results and maybe get another batch of creatures, evolved on someone else's machine... talking kilobytes, not megabytes per day here...
Of course, if this would reach Slashdot, and people started running this massively, heehee... we'd be talking exabytes :lol:
(But then one'd be able to find a sponsor, I'm sure)

Pterodactylus wrote:Team work might be fun!!!

But, on my side, I still have a lot to catch up before being reasonably "operational" and I don't have so much time... :-(

Same here, mostly reading up on the stuff on the train, heh...

enabling the breveCreature as networked version as a 'test' is a good idea, and shouldn't be too hard, most of the setup is already in place. Would give a good feel how good such a thing works, before attempting a bigger approach with more interactivity.

All that's needed is a server with a big pool, that distributes sub-pools to users, who then run for example 10 generations, send up their results, these get sorted on the server, which then sends up another sub-pool etc...
say the server runs a simple 'brevescript': generate-for the initial run- 25 random creatures per every subscribed user, so you get a nice big pool with lots of variety. Send each user 25 creatures for the first run.
As results come back, sort them in the database for fitness, deleting the bottom 25, and redistribute another randomly picked 25 to the user.
So as end user you'd have a 'regular' brevecreatures running for 10 generations, and after every cycle you get a new batch you 'optimise' by running it for 10 gens, upoad etc.
In the end, you'd get a pretty optimised creature, because the best out of a fairly big pool (depending on how big the beginningpool is, depending on the number of users...)
later, when this turns out to be working, you could add bells and whistles as you go, like subscribing into an already running sim, you gey a 'key' and the server just adds 25 new creatures in the 'megapool' (either totally new, but maybe better 25 of the cached less optimal ones, that were bound to be deleted..)
Oh, about the 'key' I mean, one could add an identifier to the genetic makeup, say NOZ runs a sim, his results are marked Creature NOZ1,2,3... gets uploaded, I get some of his results in my batch, and i run it, then, if any of them are any good, they'll end up at the top of the megapool to be redistributed to Pterodactylus, and he sees creatures on his screen called NOZ5Rxke3... etc.So he knows where the parentage comes from... (with some maxlength for the name, eventually)
So if NOz runs this 24/7 and I only 3/7 there'd be more creatures w/ a parentage named NOZ5NOZ66.... and his chances to end 'on top' are bigger. :)

Rxke wrote:enabling the breveCreature as networked version as a 'test' is a good idea, and shouldn't be too hard, most of the setup is already in place. Would give a good feel how good such a thing works, before attempting a bigger approach with more interactivity.

Yes why not, it's a start for the real fun: having creatures figthing each others... :D

Rxke wrote:All that's needed is a server with a big pool, that distributes sub-pools to users, who then run for example 10 generations, send up their results, these get sorted on the server, which then sends up another sub-pool etc...

hmmm... I wonder.... There will be synchronoztion problem, won't they? I mean, a given computer may run faster then another, M. X may join in when M. Y is testing its creature #24, and so on... So I guess the server won't wait for all clients to finish but answer a given client based on what it have on hand. Which implies that the server should also be running a bunch of creatures or should already be seeded with a few evolved creatures.

But still, I don't see the advantages... I mean let say that there is 10 clients, each running 25 creatures. If the client come back to the server every ten generations, all clients will turn out to evolve the same "species". Not funny... But still it will at least put the server in place, and there will be time after to improve the behavior.

It would be nice if different "species" may evolve... What would be fun is to have the server create a gigantestisc world and distribute part of the world among clients. Each client will evovle creatures in their part of the world and may eventually share (mate) with creatures in adjacent parts. But there should be isolated regions so that different "species" may evolve and each regions may have different terrain types to increase differences. And then, from time to time, the server launch a bunch of a "species" in a far a way region or switch population to see how a given species will adapt to its new conditions, stuff like that.

Nevertheless, its better to start simple, so just having a server that share populations among clients it's a good start IMHO.

I would like to try my hand at the capture the flag, i also thourght of a few modificatoins for it for anouther game.

i thourght each player could start with a base. there are several food sources around the "map" if a users player gets near the food it picks it up , then if it takes the food back to its base and theres enough it can create a new player on the field. if a player is hit by a apposing player it drops food, also theres a three point counter , if a player is hit three times by an apponents player it will die unless it goes back to the base to regenerate. if there base is hit lets say 30 times then the base is distroyed and the oppoenent wins the game. there could of course be obsticles or better/worse food sources. but i think think type of game would lend itself to multiplayer actoin more. Also i think i could create the game by myself (although not intill after uni). i dont think it would be much differnt from swarm really. less diffacult than capture the flag i would hazzard to guess. let me know what you think :)
Noz

Ptero, nice idea about the 'islands' and occasional 'merging-clashes' of the best ones.

re: synchronisation, I don't see the problem: if server sends out creature 24, it's out of the 'big pool', so nobody else can download it... Of course, if you run your sim on a slow computer (like mine) chances are creature 24 comes back in the big pool, at the bottom of the rung, because in the meantime all the others have got several generations of optimising... Hmmmm... No...
Maybe different pools, depending on how long you plan to run the sim... Server hands out creatures, and only goes to sorting after all results are back in.Only then it does the fitnesstest for all results and goes up to generation +1.

If you have a fast computer, you get either subsets that not have been evolved from the same pool in that generation, or a subset from a newer pool. 2 or three pools should be enough to let slower computers come in w/ results

If the computer is really waaaay too slow, or the user simply doesn't complete the run, I guess there'd be a treshold set, say several days, before his results are considered 'lost'. When he sends his results, they are obsolete, subset been given to somebody who was more active. I guess that'll need some experimenting to find the sweet spot.

NoZ, 8)

would be a great thing to evolve. Not very simple though, involving sensors and a 'brain' versatile enough to evolve different strategies, like evasive action etc.

I like it, verily. Would be like different ant colonies fourageing and fighting for food, nice.

ok it looks like weve all leaned towards the creatures demo with a server.
i think to make this intreasting we could alter it a bit , im pretty sure that the demo could be altered to the user can give a creature he/she chooses a free pass into the next round , from a menu choice. So we can keep creatures we like or think have potential :) the secound thing we can incorperate a few rules :

maxium size ie all links sizes addid together cannot be over 100.
maximum torque cannot be over whatever.
maximum speed cannot be over whater.
maximum rotatoin (dont want wheels)

other than these rules we can all modify the code , i know someone wants to make it so the limbs are semtrical , so we can all go mad , trying to make are creatures mutatoin code better. then the server will test them all. the server will breed the best 2 or three then send them to everyone. then everyone has a test group of the best and then they should be mutated by the user. if they get a distance better than the last time the server tested he or she can send it to the server and then everyone will be repopulated. this means that the server shouldnt be used to inteansely , gives everyone a bit of freedom, has a bit of compitoin and can easly find cheats becouse only objects will be transferred between us no actul code. and hopefully wouldnt be too hard to pull off but then i havent really looked into breve over the net.

So I guess it is time for work distribution...

Maybe Rxke could start the proto and people can jump in after.

On my side, I afraid I would only be useful as beta tester at the beginning... I'm mean if both of you are just newbie, then I'm not even born yet... ;-)

I've to run.
So long!

There's one snag on my side: I'm still trying to compile breve for Linux PPC! :lol:

if this is what where going to do im up for it :)
unfortantly i still got 6 weeks of uni but after that i'll be able to put alot of time and effort in.

i think two programs would need to be made , the users code and the servers code, the users code shouldnt be too hard to enforce the limitatoins, the user creature preferance shouldnt be too hard and then two menu optoins on getting new batch from server and sending best animal. then thats it for the user code,becouse the reast will be users preferance and change it however they want

the serverside code will just have to accept new uploads, test it against present best. if better than breed it with the last best then keep it in memory for when a client calls for a new animal.

like i said i can do the client side fairly easy accept for the networking bit , but i can make sure they go into two objects , so whoever wants to can just write the code to send it to the sever and recive. server side would be a bit more diffacult scince multiple users may try and do everything all at once. but all it needs to do is test it , and if better breed it.

so one person making the client without networking
one person making the server side without networking
and someone else can network them together?

anyways thats how i would think it should go down, jk and others probably have better ideas and keep in mind it'll be a while before i can actually properly start coding.then of course we dont know if we have a server yet , but im going to talk to a few people i know and see f i can manage to get hold of something resembling a server, no promices though.

Noz

hmmm...

I may be out of my mind again, but I though that breve was already supporting server/client functionalities... Nothing to build, just inherit from the NetworkServer class and implement the desired behavior...

http://www.spiderland.org/breve/breve_docs/classes/NetworkServer.html

Am I missing something here?... :?

So it seems that there is a little bit of latencies before the real start...
It might give me time to finish reading the doc and to try to beat the "Capture the Flag" winner... :twisted:

In the mean time... I'll post a question for a new walker (a Snake)...

Pterodactylus wrote:hmmm...

I may be out of my mind again, but I though that breve was already supporting server/client functionalities... Nothing to build, just inherit from the NetworkServer class and implement the desired behavior...

breve currently only supports networking to other breve simulations. What's missing is the ability for a breve simulation to submit an XML or text file to an HTTP server which would put it in the database and allow another client to grab it.

- jon

Ah. I thought it was possible :oops:

So as a workaround you could use a dedicated computer running breve, not doing any fancy simulation w/ physical objects, but just a script, that doles out the in and outcoming files?

ive been looking into network tunneling. i use a program that makes systems think there on the same intranet when really its going over the internet.

It would take me ages to figure this out though, but it would have the bonus of being REALLY secure , and have a really good connectoin speed although it wouldnt be good for like breve webpages , i think it could enable us to have a simutanious simulatoin running on multiple machines , so we could interact with each other in nearly realtime. even to the point where we could make avatars and wonder around the world together. as a bonus we can print dialog to screen so we could prehaps even make a type of "chat" where we can type messages from one part of breve and everyone else would see it.

Xlink kai is the program ive been looking at it allows Xbox's to think there on the same network when there really on the internet. i think its written in C so should be able to add it to the main breve code somewhere.

what do people think?

Hmmm... yes and no... Looks interesting but from the site I can nowhere see what kind of licence it is, so it is probably not open.
And it sounds very much geared towards xbox, so you never know which way it'll go (since it is developed 'under covers' )

NoZ wrote:It would take me ages to figure this out though, but it would have the bonus of being REALLY secure , and have a really good connectoin speed although it wouldnt be good for like breve webpages , i think it could enable us to have a simutanious simulatoin running on multiple machines , so we could interact with each other in nearly realtime. even to the point where we could make avatars and wonder around the world together. as a bonus we can print dialog to screen so we could prehaps even make a type of "chat" where we can type messages from one part of breve and everyone else would see it.

This would be a fantastic addition to breve, but I also think it would be a fantastic amount of work. I like the screensaver + HTTP + XML + php approach because most of the requirements are already there. Also, I think there will be a much wider appeal for users of the software if it's in screensaver form (as opposed to an application where the user is expected to interact directly all the time).

- jon

Mr. Correy Kowall (a.k.a ckowall here) has a pretty neat brevecluster running, complete with a 'head-node': http://brevecluster.nmu.edu/
And I asked him awhile ago if he'd considered to make it a web enabled one, and he said, in the future, but not right now, though maybe he has put some thought into it already, and has a pretty good idea what's needed/realistic?

Should I nag him again?

hey im sorry if im inaccurate but here i go

Xlink works by being anywhere on a network, it picks up packets from the xbox by knowing the xbox's IP. then it wraps it in all the stuff needid for going onto the internet. when it gets to where its going the secoundary xlink decodes the packet and then gives it to the xbox. the xbox's never know there not on the network.

I beleave this uses a virtual private network, which differs in most internet transactoins by setting up a secure route, then each client has multiple checks on it, so users cant "hack" onto the VPN. then when the clients autherisatoin goes through its allowed full acess. there is a CONTINUAS connectoin unlike html bacouse ack and nacks and al that stuff become simplified becouse its continuas , (even if its not sending anything).

now this can be used for html but its a waste becouse the startup and ending of the vpn is quite long for such small data , but if big amounts of data is sent its better but it can be used for small amounts of data, and becouse if the screen savers on we can surpose that a the user doesnt mind a bit of there bandwidth being used by breve.

back to xlink. breve can connect to other breve simulatoins on the network by using ports, same as the xbox. if xlink can trick the xbox i dont see why it cant trick breve. in theory all we have to do is make sure breve sends data on a persific port number , make sure xlink picks up on that port number and sends it to the breve server where anouther xlink will pick it up and feed it into breve. so even if we dont incorperate it into breve we could always mimmic what the xlink does and just turn that on before we want breve to communicate on the internet.

feel free to blow massive holes in my theory as im not a networking person.

Noz

I don't follow you with all this.

I'm still under the impression that we could use breve Server functionalities to implement the basic requirements:

1. Server must be able to submit population to a client
2. Client must be able to submit evolved population back to the server

It should work if both the client and Server implement a breve NetworkServer (kind of a CORBA behavior, where to establish publisher/subscriber functionalities both application must implement a Server). The MainServer will implement the receive-evolved-population method from an object of the Client, while the client will implement the receive-new-population method from an object of the MainServer.

When sending its evolved population to the MainServer (or at first contact: empty population), the Client also include its IP address and port in the data part of the Object, so the MainServer will use these information to send a new Population object back to the client... A possible issue is that the client must be correctly configured with the MainServer IP address as well as it's own IP address.

What do you think?
I'll try to do a prototype next WE if I have time and see if it works.

Also, we don't really care which client is optimizing which population. If we want to make sure that no seed is lost we might try something like this:

a. The same seed is sent to three different clients.
b. Population sent to different Clients must differ by more than "n" seeds.
c. Population breeding should occur when 1/3 of the seeds of a generation has evolved or when there is no more seed to send (generation cross-breeding is accepted).

As for security issues, I don't see the problem. I assume that most network security treat will be managed by specialized software (personal firewall), as usual. I mean, we don't need password, we don't need data encryption and stuff like that, we are just exchanging data over the net, nothing private and no secret...

I also don't see the need to protect the code against cheater, unless it jeopardize the experiment. I mean, there is no winner, no looser. Everybody will evolve the population of everybody, what's the point in cheating? And if someone cheat anyway, as long as it doesn't seriously affect the experiment, I don't see the problem.

Finally, JK would like to have the client has a screensaver. Personaly, I hate screensavers... I prefer to be able to play with it, changes paramaeters if it is possible, capture a screenshot or a movie, and stuff like that. So... OK for the screensaver as long as the stand alone client could also be run and made available for downloading.

So long.

the diffaculty is not within transferring data over a network , that seems fairly easy but its lan based not internet based. i think it pretty much only works on ports . although im ready to stand corrected , about the security aspect , i invisoin even for a simple excercise someone trying to break it by sending melicious data etc , making the server crash etc and possibly not only affecting the server but affecting everyone who connects to it. Also it would be a good thing to have in place for compititoins etc. i dont think we want to be in a situatoin where everytime we want to network a new simulatoin we will need to alter the networking alot. and for the ip tunneling (vpn) not only does it add secureity it realy increases the speed of connectoin to a server , although we might not need the speed presently becouse we'll only be sending small amounts of data , if we want to make something a bit more extravagent we dont want to be hindered by things we havent considered. I will SERIOUSLY look into this in a couple of weeks and i might even try and alter the breve source to allow networking over the internet and the whole shabang, scince we can already get networking, it should only be a matter of incorperating the extra stuff needid to go on the internet. i think this will be my main pet project after uni. although this is all conjuncture unless we get a server

Noz

breve can communicate with non-local hosts to anywhere on the internet -- no LAN required. Also, I don't think security should be a concern here, since it's just breve XML files being sent back and forth.

I would recommend, however, against the idea of having a computer submit its IP address to the server to allow the server to send it data later on. I think there will be a lot of problems with firewalls/proxies for most computers that are not intended to be servers. Not a single machine I work with (either at home or at work) would allow the incoming connection, and I think this is fairly standard these days. Instead, the client computers should periodically request an update from the server.

- jon

jk wrote:I would recommend, however, against the idea of having a computer submit its IP address to the server to allow the server to send it data later on. I think there will be a lot of problems with firewalls/proxies for most computers that are not intended to be servers. Not a single machine I work with (either at home or at work) would allow the incoming connection, and I think this is fairly standard these days. Instead, the client computers should periodically request an update from the server.

Got your point...

Well I think then that I'll take a deep breath this WE and dig in the C code behind this NetworkServer class just to see what is going on...

hmmmm...

I've been reading the code in detail (mainly network.h, breveFunctionsNetwork.cc, breveFunctionsSteveObject.c) and from what I understand so far, correct me Jon if I'm wrong, aside from http commands, breve only support that clients send objects as xml file. Once the object has been received, the server doesn't (and doesn't have ways to) send something back.

In other words, there is no support for sending back a new population to the client.

I saw that there are room for two other request types (NR_UPLOAD and NR_METHOD). These request doesn't seems to have been implemented.

So, I guess that we could implement the NR_UPLOAD so that the client could upload the new population, or the NR_METHOD if we want to be more fancy about it. Right?

Do you have any requirements/planning for these implementation?
Did I guess right? What's the deal?

Assuming we should implement such client/server functionalities, it seems managable... with a lot of testing... :?

Forget what I've told about NR_UPLOAD... Was very tired last night when I wrote this post...

My best guess is that we should add a new request type NR_GET_POPULATION, for example (feel free to pick a better name, english is not my first language). On the client side, the following methods should be added to the Object class:

get-population-over-network to hostName (string) on portNumber (int)
set-population-upload-recipient to newRecipient (object)

and the client controller object (or object designed by the set-population-upload-recipient method) should implement the method

+ to accept-population-upload from-host h (string):

This method will do what must be done to insert the population and run it.

On the server side, the server (PopulationServer, for exmaple) should inherit from NetworkServer and should implement the method

+ to send-population-to-client of-instance i (object) to-host h (string):

Something like this, if you get the picture.
Any comments?

So basically, we want to start developing various core-war type competitions? This sounds great, and I would love to be involved.

~Moto

I have modified the neural network class slightly and use it in all sorts of simulations based on the a tricked out braitenberg vehicle code...I actually should post my new more modular code....Hopefully you can use or gain insights from this code.

http://brevecluster.nmu.edu/files/A1.tz
http://brevecluster.nmu.edu/files/Braitenberg4.tz
http://brevecluster.nmu.edu/files/NeuralNetwork3.tz

Yeah so this stuff is kinda intended to be modular... hence possibly applicable to your project. Note that all the braitenberg appliances now inherit from segment which is cool well because object oriented design is cool. Also I will eventually adapt the manipulator class to inherit from directed graph.tz.

Feel free to join in! :-D

Actually, I'm afraid that the project is on hold due to a lack of resources, time and "knowledge". But if you can provide time/ideas/knowledges even jokes (;-)), you are welcome.

Maybe we should try to find out who can do what and tries to settles thing on the fly... So, just to have an idea and try to organize things...

Who can provide time (and how much)?
Who have a good knowledge of breve programming?
Who have a good knowledge of AI concepts?
Who just want to chuck ideas out when they come (which is a good thing too)?
Who can provide a server??? :? -- just trying...

Personally, I don't have a lot of time and my knowledge in the field and breve programming are terrible (though I'm trying to improve both :oops:), but I'm very interested and I'll try to do my best considering my own limitations.

Thanks for your classes ckowall (did not have time to peek in yet, but it's on my list)...

In the context of the previous discussion, I think it's important to note that if local populations are not recipients of migration, a 'concentrator' population will narrow to a solution that does not always benifit from competing conventions evolved in those local populations. However, on the other hand if the members of a concentrator population are migrated back to the local simulations, the convergence of the population is much much slower and performance of induviduals, globally, is comprimised. Or at the least that is what I have observed on my cluster. It might be worth noting that when I used the built in accept and send over network code, I could not get the client side to successfully upload induviduals more than once per restart. So, in the most recent versions I have switched to using the very reliable save-as-xml feature to save to a NFS folder which in turn can be accessed from the other nodes in the cluster. This is probably not so usefull for the project under discussion here but it might prove helpfull if someone else wants to undertake a cluster project.

ckowall wrote:In the context of the previous discussion, I think it's important to note that if local populations are not recipients of migration, a 'concentrator' population will narrow to a solution that does not always benifit from competing conventions evolved in those local populations. However, on the other hand if the members of a concentrator population are migrated back to the local simulations, the convergence of the population is much much slower and performance of induviduals, globally, is comprimised. Or at the least that is what I have observed on my cluster.

Hmmm... Not familiar with all words... What do you mean by "concentrator"? :roll:

Regarding your comment on the availbale NetworkServer functionalities, from what I've seen in the code (I mean the source code), the server is not built to share populations. You can send population to it, but I did not see mechanisms to send it back... So, my guess is that it should be possible to implement a new "breve level" communication protocol that let the client send a population to the server as well as ask for a population (after all, it's just a TCP/IP server underneat). This part, I'm pretty sure I can handle it, if I can find time...

The NetworkServer should be able to support receiving a population of agents, as long as you send the population by encapsulating the agents in another object and set up the dependencies correctly as described in the documentation on networking and object archiving.

I still like the idea however of the server not being a breve simulation and instead being implemented using PHP. The downside of that, of course, is that breve *doesn't* support that at the moment.

- jon

in 15 days i will start creating what ever needs to be created except for networking. i was thinking of making a simple simulatoin and im going to try and use a computer ive got in my house as a server. im hoping to have brevecli running in the background continuasly that other people can communicate to BUT this is not a promice.

maybe try and get capture the flag working so people can upload there player and it will take part in a tornamate

jk wrote:The NetworkServer should be able to support receiving a population of agents, as long as you send the population by encapsulating the agents in another object and set up the dependencies correctly as described in the documentation on networking and object archiving.

Hmmm... Sorry Jon, but I'm not sure I'm following you...

Let's forget about the archiving part for now, which, in my understanding, only defines what exactly will be sent.

I know that NetworkServer can accept population, the problem is it can't give back population. Even if the method "accept-upload" is implemented properly (BTW the accept-upload method of GeneticAlgorithm is missing in the doc), there is no mechanisms that allow the server to sent back a population. Do we agree on this?

If I'm not wrong (and I'm ofently wrong) we need to create a new network request type that will let us say in the accept-upload method of the server: "hey, please send this population back to the client". This part is missing in breve server/client "protocol in my understanding.

What say you?

Oh, I'm sorry -- you are correct. So yes, we'll need to implement a "get-object" type of request for NetworkServer. To keep it as general as possible, I guess the NetworkServer should be sent an arbitrary string, and then let it interpret that string however it wants to figure out what object to send back.

- jon

OK, fine then.

If you don't mind, and assuming you don't plan to implement such a functionality on your side, I would like to give it a try.

I might go slowly (don't have so much free time), but hoppefully I may find enough time to make the first draft and submit for comments/suggestions/feedback. Especially the kind you just made (name of the method, type of arguments, how it should be to make it more general...)

I have a little network at home so I'll be able to do limited tests and debugging...

...I'm eager to start... :-D

I think such a change would be a great addition to breve, and I'd be very grateful if you wanted to take a shot at implementing it.

Sometime in the next couple of days I will write up some quick notes on some of the functions you might need and where to find everything!

- jon

jk wrote:Sometime in the next couple of days I will write up some quick notes on some of the functions you might need and where to find everything!
OK thanks! I'd appreciate that. If you have a list of requirements it is also welcome.

I'm setting up my dev environment... Just finished compiling breve 2.4... It works!!! :-D

Though ODE gave me a hard time... But now that I have ODE source code on hand, I might have a chance to look into details on this mu factor... :?

Please keep us updated on progress involving the new networking class. I, for one, would love to begin development of a competition environment! It should be great fun playing core war with our simulations.

Moto

OK, I'll do that.
I think I should have time this WE to start it (my dev environment being now ready).

My intention is to post the modified C/C++/breve code so that everyone who can test it, could also give feedback.

Sorry I'm a little late getting you these notes.

Currently, breve handles network uploads in kernel/internalFunctions/breveFunctionsNetwork.cc. At first I was going to suggest a distinct download request, but now I'm thinking, instead of implementing a separate download request, what if the existing upload code could optionally send back an object of its own?

In this way, the current upload functionality is used to handle the request. The uploaded object would be a empty object except for a simple line describing its request (or, at the user's discretion, more complex information). We could create a special class for this called NetworkRequest, or something to that effect.

Every breve network interaction would thus consist of initiating a connection, making a request and (optionally) getting a reply. This will require very little new code (compared to the alternatives). I also really like this because using breve objects for the communication means that we can modify the underlying protocol while preserving the high-level behavior. This is good because on my TODO list is to switch from TCP to UDP for faster networking.

So here's the process as I see it:

1. Client makes request to breve server by sending an object (already implemented)

2. breve internal code decodes the object (already implemented)

3. breve internal code makes a call to "accept-upload" (already implemented)

4. Server's implementation of "accept-upload" can inspect the object and optionally return another object. Imagine for example, that the accept-upload function understands "get-population", "get-fitness", etc.

5. If an object is returned by "accept-upload", it is XML encoded (see the function stXMLWriteObjectToStream in steve/xml.cc).

6. The XML is returned to the client.

7. The client decodes the object using stXMLReadObjectFromString (steve/xml.cc) and returns the object as the return value to the method which initiated the request.

All in all, I'm shocked by how little work this ends up being by using the breve objects as the requests -- I was expecting it to be a lot more!

Have a look at the steps I've outlined and let me know what you think. I'd be happy to work with you on this and provide whatever support I can.

- jon klein

Thanks for the list. Seems fine...

Sorry, to be late, let say my WE has just started... Went for a walk in Jura mountains yesterday, so did not have time for breve... :?

So, if I do follow you, you are suggesting to create a new breve Class, NetworkRequest, that inherit from the Object Class. This Class will then have a way of saying to the NetworkServer to please send an object (of the same derived Class) back.

I wonder if instead it wouldn't be better to add a new optional argument in the send-over-network method of the Object Class:
Object.send-over-network to hostname (string) on portnumber (int) wait-for-object wait = 0 (int)
[Note, for now on, I will always prefix method names by the Class that implements it.] By default, the code is not waiting for an Object, unless wait is different than 0. In this way, the code would be backward compatible and easy to upgrade from a breve developer point of view. And, the behavior will propagate to all classes deriving from the Object Class: i.e. all breve Classes.

Or the Class name of the Object the client is waiting for could be passed as a string:
Object.send-over-network to hostname (string) on portnumber (int) wait-for-object class = "" (string)
Empty string, means "I'm not waiting to get something back". In this way, a client may send an Object and wait for another Object. I see no use for now, but it is a more general behavior...

Or the string may be commands as you suggest ("get-population", "get-fitness", etc.) which, I assume, are triggering a method of the same name in the server receiver class...

Hmmm... I wonder... It seems to me that, assuming I did get you right, this solution implies that we need to implement a new generic method in the server receiver Class (which may be any Object, so in the Object Class) that will be called by the breve developper specific implemantation of "get-population" or "get-fitness" to finally send something back:
Object.send-back-to-client object i (Object) at-host h (String)

All in all, I can't say which implementation is better. I'll let you pick one (or propose another one), you have a better feeling of what fits better in breve coding style.

Also, just to be clear, if you think you can implement this quickly and wish to do it, there is no problem with me, I won't be offended (even if I've already started to work on it). I really don't want to slow things down. So, just let me know.

That having been said, let starts the real work now... well, as much as I can, considering that there are still open issues...

I've modified the NetworkTest.tz a little bit... Code follow.

The modified code creates three agents that are x-shifted one from the other by 5 units. Agents are moving on the y axis from -5 to 5 reappearing at the other edge once they've reached the y-limit. Agents are sent to the Server, one at a time with a delay of 120 time units.
I'm observing a strange behavior on the Server side... Whites circles appears on the screen on a fix position and stays... Probably something I don't really understand on the way accept-upload creates objects...

Also, it reminds me the post of ckowall who was saying that he couldn't get the client to upload individual more than once per restart. I also observed odd behavior if objects are sent with very short delay... Makes me wonder if the NetworkServer implementation was thoroughly tested? :roll:

Here is the code, case you might find it handy. The idea is to improve this class to be able to test new functionalities when they'll be available.

BTW, I use the same class for the Network and the Client, though they must both run on separate computer (or breve instance). To toggle between Server/Client behavior, you must set the I_AM_THE_SERVER flag appropriatly, as well as the BREVE_SERVER_IP.

Time for lunch... ;-)

#!/Users/jk/dev/breve/bin/breve --version

# include some frequently used classes and specify common classpaths

@path "classes"
@path "lib/classes"

@use Mobile.
@use Control.
@use Stationary.
@use NetworkServer.

@define I_AM_THE_SERVER 1.
@define BREVE_SERVER_IP "192.168.1.134".
@define BREVE_SERVER_PORT 30304.
@define Y_LIMIT 5.

# tell the engine the name of the controller

Controller myController.

# implment the controller object

Control : myController {
    + variables:
        agent1 (object).
        agent2 (object).
        agent3 (object).
        current (int).

    + to init:
        if I_AM_THE_SERVER: {
            new NetworkServer listen on-port BREVE_SERVER_PORT.
            print "NetworkServer is ready...".
        } else {
            agent1 = new Agent.
            agent2 = new Agent.
            agent3 = new Agent.
            
            agent1 init-with x-shift -5.
            agent2 init-with x-shift 0.
            agent3 init-with x-shift 5.

            self schedule method-call "send" at-time 5.0.
        }


    + to iterate:
        super iterate.

    + to send:
        if current == 0: {
            print "Sending $agent1 over Network to", BREVE_SERVER_IP, "at port", BREVE_SERVER_PORT.
            agent1 send-over-network to BREVE_SERVER_IP on BREVE_SERVER_PORT.
            current++.
            self schedule method-call "send" at-time (self get-time) + 120.0.
        } else if current == 1: {
            print "Sending $agent2 over Network to", BREVE_SERVER_IP, "at port", BREVE_SERVER_PORT.
            agent2 send-over-network to BREVE_SERVER_IP on BREVE_SERVER_PORT.
            current++.
            self schedule method-call "send" at-time (self get-time) + 120.0.
        } else {
            print "Sending $agent3 over Network to", BREVE_SERVER_IP, "at port", BREVE_SERVER_PORT.
            agent3 send-over-network to BREVE_SERVER_IP on BREVE_SERVER_PORT.
        }

    + to accept-upload of theObject (object) from host (string):
        print "received upload of $theObject from $host".

        if ((theObject get-sub) get-list-string) == "cat": {
            print "Upload of subordinate object successful".
        } else {
            die "Error dearchiving networked object".
        }
}

Mobile : Agent {
    + variables:
        sub (object).
        subShift (vector).
        xShift (float).

    + to init-with x-shift x (float):
        sub = new SubordinateAgent.

        xShift = x.
        self add-dependency on sub.
        self set-velocity to (0, -.5, 0).
        subShift = (0, .5, 0).
        self move to (xShift, Y_LIMIT, 0).

    + to iterate:
        if (self get-location)::y > Y_LIMIT: self move to (xShift, -Y_LIMIT, 0).
        else if (self get-location)::y < -Y_LIMIT: self move to (xShift, Y_LIMIT, 0).
        sub move to (self get-location) + subShift.

    + to get-sub:
        return sub.

    + to dearchive:
        result (int).

        result = (super dearchive).

        self set-velocity to (0, .5, 0).
        subShift = (0, -.5, 0).
        self move to (xShift, -Y_LIMIT, 0).

        return result.
}

Mobile : SubordinateAgent {
    + variables:
        l (list).

    + to init:
        l = { 1, "cat" }.
        self set-color to (1, 0, 0).

    + to get-list-string:
        return l{1}.
}

I think I've a good idea now of what I must do.
I will basically follow the line of what I've already sketched base on your suggestion:

1. Update the steve/breveFunctionsSteveObject.cc class, namely:
A. Declaration of sendXMLObject should return AT_STRING instead of AT_INT.
B. stNSendXMLObject should return a string (can't see what's the use of the brEval *target argument BTW).
C. stSendXMLString should return a string it should also read from sockfd for the Server answer... (so must have a default answer that says EMPTY_OBJECT, otherwise we need a trigger in the call)

3. Update kernel/internalFunctions/breveFunctionsNetwork.cc:

Method brHandleConnection should be able to send back an XML object assuming the accept-upload method return an object, or an EMPTY_OBJECT if the upload method return nothing so that the client does not wait indefinately.

4. Provide a way for breve developper to return XML string object from the accept-upload method (I did not check this part yet).

Took me more time then what I've expected digging into the code... I'll stop here for today.

I'll go along this line unless you have other suggestions.

I had a chance to work on the server tonight...

I should have finished the "obvious" parts soon:

- Client always listen for a feedback of the server,
- Server always send a Feedback to Client NR_XML requests.

Next step:

1. Client: Handling of the (possibily NULL) object returned by the sendXMLObject function. - I have no guess yet...
2. Server: Have a breve function that will XML encode an Instance of an object and return it as a string

So, to summarize, here is the implementation I'm following:

1. Client makes request to breve server by sending an object
- I have changed the sendXMLObject prototype, it should now return an object.
- Finally, stNSendXMLObject and stSendXMLString returned values are unchanged (and I do understand the use of the "brEval *target" now... :shock:)

2. breve internal code decodes the object (no change)
3. breve internal code makes a call to "accept-upload" (no change)
4. Server's implementation of "accept-upload" must return an empty string or an XML encoded object.

- Need a new breve method to do this... (TBD)
- I know, I don't implement parsing of query as you suggest, but as a first implementation I think it should fulfill basic requirements...

5. breve internal code send an empty object or the object returned by the accept-upload method to the client. (- in progress)
6. The client decodes the object using stXMLReadObjectFromString (steve/xml.cc) and returns the object as the return value to the method which initiated the request. (- in progress)

breve codding is not always obvious... So many layers... But it is fun...

Hi,
looks like you've got a good handle on the process here.

If you haven't already, you can create a SourceForge account and I can give you write access to the subversion repository when you're ready to commit your work (or work-in-progress).

I've made some clarifications below. Let me know if there's any other info I can provide.

Quote:
- Finally, stNSendXMLObject and stSendXMLString returned values are unchanged (and I do understand the use of the "brEval *target" now...

The plugin documentation covers this area. Basically, the "target" is the brEval return value of the breve internal function. Both the arguments and the return value are encoded as brEvals.

Quote:
I know, I don't implement parsing of query as you suggest, but as a first implementation I think it should fulfill basic requirements...

Won't this be handled automatically? The accept-upload object is already getting the uploaded object, so won't it be up to accept-upload to decode what the user wants? There should be no special treatment here. It's a simulation-time decision, I think....

- jon