Developing with JXTA

What does the PSE Membership do for you

2005-04-27T11:31:00.000-07:00

It makes no doubts that the pse membership is a great improvement compared to the "old" password membership, as for instance, the encryption used in the PSE membership is much stronger that the Password membership one.
However, my experience with the PSE membership is that I've only be able to manage a system where knowing the PSE Group's keystore password it the only required field to be able to be granted peer group membership.

The PSE effort is really a step forward in more secure peer to peer communication. But in my opinion its use for Peer Group Membership could be improved, by making a better use of its underlying X509Certificate / private keys technology.
For instance, instead of sharing the peergroup keystore password among the peers authorized to join the PSE Peer Group, we could instead base authentication and peer group membership authorization on whether the edge peer X509 Certificate has been signed by a "trusted Source" (e.g: the Peer Group X509Cert/private key).
In addition to be providing individual distinct authentication tokens, it would remove the need to have the peer Group private key embedded into the peer group advertisement. I feel that publishing the private key, even strongly encrypted, could be a liability.

Joining a Peer Group with PSE Membership

2005-04-26T14:49:00.000-07:00

This posting deals with the comments I have regarding Joining the SAXTA Peer group (using PSEMembership).
To manage to join the saxta group, i reused extensively the code provided in the JXTA plaform, especially the net.jxta.impl.membership.pse.pseMembershipTest class. I want to thank JXTA for providing it.

Comment 1

The first step into joining the SAXTA group is getting a instance of AuthenticationCredential using the following constructor:
AuthenticationCredential(PeerGroup peerGroup, String method, Element indentityInfo)
Let's discuss about the parameters of this constructor:

The "peerGroup" parameter is self explanatory, it refers to the peer group that we are trying to join.
The "method" parameter is more confusing, the java doc says: "The authentication method which will be requested when the AuthentiationCredential is provided to the peergroup Membership Service." Valids for the "method" parameters are undocumented and to find out what they are you have to look into the source code of the AuthenticationCredential and PSEMembershipService classes. The String "StringAuthentication" for the method parameters makes the constructor return a non-interactive AuthenticatorCredential instance, which is what I wanted.
Last The Element "indentitityInfo" parameter is described as "Optional additional information about the identity being requested which is used by the authentication method. This information is passed to the authentication method during the apply operation of the Membership Service". I'm still unsure of what it is, but tt works when you pass the AuthenticatorCredential constructor the value null. Cf comment 2 and 3 previously posted on this Blog (April 25th).

So, finally here is what worked for me:

AuthenticationCredential authCred =new AuthenticationCredential( saxtaGroup, "StringAuthentication", null);

.

More about my experience in PSEMembership

2005-04-26T09:36:00.000-07:00

I am resuming today the blog I was writing yesterday about my experience the JXTA's PSEMembership

Comment 4:

In the PSEMembershipService class, the "join (Authenticator authenticator)" method (see code snipplet below) not only makes sure that the Authenticator provided is "completed correctly" (isReadyForJoin() method), :

public Credential join( Authenticator authenticated ) throws PeerGroupException {

if( this != authenticated.getSourceService() ) {
throw new ClassCastException( "This is not my authenticator!" );
}

if( !authenticated.isReadyForJoin() ) {
throw new PeerGroupException( "Authenticator not ready to join!" );
}

but also that the but also that the Authenticator passed to the PSEMembershipService is the same as the one that was provided initially:

if( this != authenticated.getSourceService() ) {
throw new ClassCastException( "This is not my authenticator!" );
}

Such may prevent "Authenticator" spoofing to some extent.

However since the Membership class to be used in a given peergroup is only described by it class name in the Module Impl Adv
(See XML Element extracted from Membership Module Impl Advertisement below)

<svc>
<jxta:mia jxta="http://jxta.org">
<msid>
urn:jxta:uuid-DEADBEEFDEAFBABAFEEDBABE000000050306
</msid>
<comp>
<efmt>
JDK1.4.1
</efmt>
<bind>

V2.0 Ref Impl
</bind>
</comp>
<code>
net.jxta.impl.membership.pse.PSEMembershipService
</code>
<puri>
http://www.jxta.org/download/jxta.jar
</puri>
<prov>
sun.com
</prov>
<desc>
Module Impl Advertisement for the Certificate Based Services
</desc>
</jxta:mia>
</svc>

Nothing prevents me to write a rogue net.jxta.impl.membership.pse.PSEMembershipService Class, and replacing the orginal one with the rogue one on my local peer.
This would allow my local rogue peer to join a PSEMembership managed peer group.
http://www.jxta.org/download/jxta.jar file could be signed, preventing tempering, but my experience is that you can "custom" the jxta.jar file without any consequences. So should be signed, the code does not check for tempering.

Comment 5:

I used a PSEConfigAdv instance to put together the Peer Group Advertisement parameters needed for the PSE Membership. It works well if you specify only the following PSEConfigAdv Parameter the Certificate Chain or Certficate ; e.g:

pseConfigAdv.setCertificateChain(x509CertificateChain);

and the Private Key; e.g:
pseConfigAdv.setPrivateKey(nodeGroupPrivateKey,saxtaGroupPSEMembershipPassword);
Now should you provide other parameters such as the "KeyStore Type" and "KeyStoreProvider" the Peer Group advertisement would look like:
...

<svc>
<mcid>
urn:jxta:uuid-DEADBEEFDEAFBABAFEEDBABE0000000505
</mcid>
<parm type="jxta:PSEConfig" jxta="http://jxta.org" keystoretype="jks" keystoreprovider="SUN version 1.5">
<rootcert>
<certificate>

...

The 2 added attributes will actually make the JXTA throw and exception:

Exception in thread "main" java.lang.reflect.UndeclaredThrowableException
at net.jxta.impl.peergroup.GenericPeerGroup.loadModule(GenericPeerGroup.java:818)
at net.jxta.impl.peergroup.GenericPeerGroup.newGroup(GenericPeerGroup.java:1304)
at net.jxta.impl.peergroup.PeerGroupInterface.newGroup(PeerGroupInterface.java:290)

This error took me a [very] long time to find out as the error message make you think that the ModuleImplementation Advertisement is the problematic one, not the peer group advertisement.

Comment 6:

Below is the valid PSEConfig Advertisement part of the Saxta Peer Group. This PSEConfig Advertisement what created with the following commands:

PSEConfigAdv pseConfigAdv = (PSEConfigAdv) AdvertisementFactory.newAdvertisement(PSEConfigAdv.getAdvertisementType());
pseConfigAdv.setPrivateKey(nodeGroupPrivateKey,saxtaGroupPSEMembershipPassword) ;
pseConfigAdv.setCertificateChain(x509CertificateChain);

Note that what was passed to the pseConfigAdvertisement was a CertificateChain (an Array of certificates) containing the Certificate Authority's X509Certificate (at index 1) as well as the signed PeerGroup X509Certificate (index 0).

The correspond advertisement Element is as followed:

<Svc>
<MCID>
urn:jxta:uuid-DEADBEEFDEAFBABAFEEDBABE0000000505
</MCID>
<Parm type="jxta:PSEConfig" xmlns:jxta="http://jxta.org">
<RootCert>
<Certificate>
MIICJjCCAY+gAwIBAgIBATANBgkqhkiG9w0BAQUFADBJMRUwEwYDVQQKEwx3d3cuanh0YS5vcmcx
ETAPBgNVBAMTCFNheHRhLUNBMR0wGwYDVQQLExRFQzgwREE5RDdCNkI4Mjk1MkU2NjAeFw0wNTA0
MDQxOTAxMTJaFw0wNTA4MDIxOTAxMTJaMGkxFTATBgNVBAoTDHd3dy5qeHRhLm9yZzExMC8GA1UE
AxMoVU1EIFNheHRhQ29tbXVuaXR5IE5vZGUgR3JvdXAgTWFuYWdlci1DQTEdMBsGA1UECxMUNTBD
OUIyRDg3Rjk4NzhGNkI3ODIwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAKJS038q+gmKGjff

kv22pt8AuMIzqkU5ciofp0ZgavgNSLBmCwbuEq+Z2LvguBMU2zzHlVGH2xBjjFs9ZaO5chEqztyd
X4lrGj6r1aiY07JhTrKehQ/sIzk4ZK7KZ7ZvzAQECcIfuz/leBVeRh0pVGtvXRr8f8Km393cuPb2
vqRHAgMBAAEwDQYJKoZIhvcNAQEFBQADgYEALZPcRiYf4BBaZYBn0oltKNWiHKgSgHcxnWUL2S+K
hrGpjeDXbUH9MbUPPrMZ39EjsIL/qU8lTt05SDMK05Y18JdX3lXQZ1Q+E9hm/MB8pFHYX8AGLCMh
YwmnjCUV0uv/oxwOR7+a2v+JKL+GoGT/C/OHQ2rJfDJ/pwoZVNd7CbM=
</Certificate>
<EncryptedPrivateKey algorithm="RSA">
MIICoTAbBgkqhkiG9w0BBQMwDgQI8XQZohplhAECAgH0BIICgGiECYe6x10txBR7Zf6vHsG5O5q1
8+4OO4ns7f0RpYPqHGYW6snPZrg4DoZ2lbZajGz5xXaboS0W+EzGWzFFTzSL4RfTjvQIx7+uyWV8
In9CLO0Spq2AdQV1tqAEB1lPmCe/sAuFZGeVQzaJrmut2ibyNZvGSLFjQXJ1rs/8jgOUmerK/JMg
5ksIwm0TsNMiEDbE+CAwhWfdgqHbBA+dmqV+E7KWJrbhMMqnUM6a6m42b5ZoMjDQlpC95fsZtpE6
gMY1oTOubYPHgs+p9cSDe+pcQDV19+mjXPqsbVtqWrPezn1+xMpj2INLM2STnemVXD4CQb4c+tXc
viRNlYr1gyQQ/A262wPx3kI9sh7HXN2ZBO2xrlWCBxf8/AedoMpba0bw/Y6LkxcSvCEDTM4WtVFT
d2Dh4W3jqRt62rXqpq+vzjLL0aemQygBUku/WAUHn+qk7HRTJamRdiOPYKd4ZDxkyMedXlkwIocQ
d35yfQ5fvNbOe0PLkRXnjid7Af2S7XNfs8eTZSCXhzxV1wuEH6jRiMgW9uzpp2EJjdugrboe8gty
C00HoP0U02usUVKqoJtMS0dX/PJHbIMLXvXPqJnZ+3TKpWpxxNC2I3qurhw2k4RH6LGSqK6Qsc3Q
HSQBvoJxNaN0AG60916giLwSIRJIgKIAtyxNvhuBkZuEHN5po1bT62zQ2tGQjU0/cLvuJ/6eH6Jz
G17L1tRSWmjtLLMIdbQ90iqBMJQf0YSs5hJ8UJGW6ST3aMnOP9Rf0Ge+QK1GRjuGoI0KEo9xSom6
iUqEIzutFQYiCCm3P2M/R/FQZd0KsigsaiIA0YnHNy3pZKrrzZOsoSczyhyIGnHX/gA=
</EncryptedPrivateKey>
</RootCert>
</Parm>
</Svc>

The certificate chain has been truncated to the Peer Group X509Certificate (the CA X509 Certificate has been discarded), which confirms the comments in the JXTA code that the PSE Membership does not support certificate chains.

It actually is an issue for me as I rely on a "self signed" certificate authority to identify potential rogue/unauthorized super peers: Valid Super Peers have their Certificates signed by the CA.

Sharing my experience in working with the PSEMembership

2005-04-25T16:41:00.000-07:00

Introduction

This posting is an ad hoc set of personal comments about the experience I've had so far in using the PSE Membership. It refers to the previous posting as for the goals and steps that I am trying to follow.

All right, let's start, readers, please don't be too upset by what below. I'm not an expert SW engineer, so It may be inappropriate for me to post these comment, but I just hope it helps ...

Comment 0.0:
It may have changed, but the building of the JXTA platform is a little funky. There is no way to compile the platform with Eclipse for instance as some java files (for monitoring if I recall) are generated by the JXTA build.xml ant file. The only way to build the platform is to use the JXTA ant file. So I need to use the jxta.jar library when developing my application. That is fine until I need to look into the JXTA platform source code. I can't do it from eclipse, I need to use another editor for editing the platform source code.

Comment 0.1
Again, it may have changed, but on the www.jxta.org web site front page there is no link to download.jxta.org which was confusing when I first tried to get the binaries of the latest stable release. If you want the latest platform "you've got" (again not really, but a link to "download.jxta.org" is hard to find) to get it through CVS which could discourage some users.

My Experience with PSE

Generating and signing certificates.

Step 1 through Step 2.2 were no problem. Creating a Self signed Certificate Authority, certificate Signing Request, and actually signing them, checking signatures is actually fun !
I used JXTA FileKeyStoreManager, and PSEUtils classes to manage my CA as well as my "Super Peer" keys and certificate chain. I got a big help by looking at the pse.* command of the JXTA Shell. It really helped. Thanks to the JXTA community for suggesting it.

Comment 1:

I would just regret that JXTA does not follow any standard as to the use on the X509 Distinguish Name. At least JXTA should document its use of the Certificate fields such as its use of JXTA ID within the DN, or the forced appending of "-CA" .

Generating and the SAXTA Peer Group and Peer group Advertisement.

I actually under evaluated the time I would spend on that task. Since I had experience with "custom" peer groups and ModuleImplementation, I though I had an advantage. I'll have the following comment on my experience in doing so:

Comment 2:

My vision of the JXTA platform is that it provides some JAVA Abstraction to the JXTA XML protocols. Specifically, the platform is a layered application which upper layer provides fully object oriented and abstracted view of the JXTA concepts. I actually would consider the JAVA platform to have about 4 different abstraction layers (From higher to lower):

The Concept / Abstracted layer. E.g: A Peer Group
The Advertisement Layer. E.g: A Peer Group Advertisement
The Document Layer (or XML Layer): E.g: The StructuredDocument corresponding to the Peer Group Adv
The Transport Layer. E.g: http, tcp ...

Each layer is "self contained" and provide a specific representation of the JXTA protocols. The higher the level, the most abstraction and ease of use. The lower level, the most thing that is left to be managed, but also the more flexibility. Naturally there are some junction points between 2 consecutive layers. For instance, you can switch from Peer Group and Peer Group Advertisement there is peerGroup.getPeerGroupAdvertisement() and a newGroup (peerGroupAdvertisement). I understand that JXTA would not consider a good practice to the mix of layer such as newGroup (peerGroupStructuredDocument).

However I have not been able to do otherwise while assembling a MembershipModuleImplAdvertisement. I have not seen a work around to the following syntax:

certificateMembershipModuleImplAdv.setParam((Element) certificateGroupParamAdv.getDocument(MimeMediaType.XMLUTF8));

Comment 3:

I personally am not found of having a "null" value being a valid in a method. For instance, I am not found of the following type of syntax: peerGroupDiscoSvc.getRemoteAdvertisements(null,DiscoveryService.GROUP,null,null,1000);
or
peerGroupDiscoSvc.getLocalAdvertisements(DiscoveryService.GROUP,null,null);
For the least it forces the user to look into the javadoc to try to figure out what are these parameters to learn that a null value is acceptable as default. More often that not, it also makes the user look into the source code itself to try to get an answer.

Looking into the PSE Membership

2005-04-25T15:08:00.000-07:00

OriginalGoals and Expectations from the PSE Membership.

As described on a Email Sent to the JXTA user mailing list on March 23rd 2005, what I was trying to achieve with the JXTA security is to create a certificate based authentication peer group. This peer group would allow or deny peers based on whether their X509 Certificates were signed by Trusted Peers. Also the super peers would also have their certificates signed by a self signed "Certificate Authority". This would prevent "rogue" Super Peers.

To make it easier, these trusted peers would assume the following charges:

Super Peers
Advertising the "Certificate" peer group
Rendez vous and relay services for both netPeerGroup and the "Certificate" Peer Group

For the rest of today's posting I'll call this Certificate Peer Group, the "SAXTA peer group".

In short, the Certificate Trust Chain would be:

Certificate Authority (CA) --Signing--> Super Peers --Signing--> edge peers.

Envisioned Steps to Achieve these goals

The Steps I was planning to go through to get there were:

Create a self sign CA.
Create a certificate based authentication peer group whose certificate is signed by the CA. That includes:

Generate a X509 + Private Key Pair for the peer group.
Have the CA sign the X509 (by emailing a cert request for instance). I saw that being an "off line" process.
Build a Membership Service based on the signed X509 + Private Key Pair. I was expecting the PSE Membership to do so.

Having edge peers joining the peer group via a certificate Base authentication

Having the Edge Peer generate a X509 + Private Key Pair for peer group membership.
Having the Peer Proup manager sign the edge peer certificate. Again, I saw that being an "off line" process.
Have the edge peer contact a super peer, exchange certificates:

Having the edge peer verify that the super peer certificate is valid using the CA's X509 (The CA X509 is bundled into the application).
Having the super peer verify that the edge peer membership application is valid by looking at its certificate and see if is it signed by an authorized super peer (again by making sure the super peer's certificate has been signed by the CA).

Finally, having the Edge Peer join the peer group

Getting Started

I was very happy to receive a reply to my March 23rd Email from the JXTA user list saying that this approach could make sense. Yes ! Let's get to work !!!
I was actually expecting the PSEMembership to provide pretty much to all these functionalities. Otherwise I told myself, what would have been the point of using X509 Certificate, key stores, root certificates, etc. in the PSE Membership?
I could not wait to finish a prototype, then come up with a "good" design and re-implementation. I would then turn it into a "tutorial" and propose it to replace the "Secure Peer Group" example in the JXTA Programmer's Manual I wrote years ago. This stuff is really old and given the encryption method, it does not provide actually much added security.

A month later, I am still in the prototyping phase. Even though, I don't have the opportunity to actually work on the coding part more than 30-40% of the time, it is still much more frustrating that I though...

Presentation of the SAXTA project

2005-04-21T09:57:00.000-07:00

Project SAXTA

A new, loosely coupled network dedicated to the open sharing of high resolution earth observation data sets.

Mission and Objective
The mission of the SAXTA development project is to establish a new distributed system for the sharing of high-resolution (~30m) satellite datasets, such as Landsat and ASTER data. The objective is to dramatically improve access to and distribution of non-copyrighted high-resolution satellite data sets to the science, applied science, and education communities.

Basis: NASA's Open Data Policy
NASA’s Open Data Policy provides the foundation for the system’s distribution model: NASA is committed to the full and open sharing of Earth Science data obtained from U.S. Government-funded and -owned systems with all users as soon as such data become available. All Earth System Enterprise missions, projects, and grant proposals shall include data management plans to facilitate implementation of this principle [1]. As a result of this, NASA’s Earth Observation data are not copyrighted or otherwise restricted in distribution whether the data is purchased or distributed for free. The SAXTA system will greatly extend access to and distribution of these data and is completely aligned with NASA’s Open Data Policy.

Statment of the Challenge
NASA currently sponsors a large number of projects that acquire and utilize high-resolution non-copyrighted datasets such as Landsat and ASTER. Other agencies and organizations also sponsor a significant number of projects. These distributed archives cannot currently be discovered or their inventory searched in any practical way, despite the fact that these datasets hold substantial potential value within the larger scientific and education community. The “owners” of the data would welcome more extensive usage of their holdings, as well as the ability to obtain data from others. Some of the larger projects such as Landsat.org (MSU), a University ESIP and Mid West RESAC partner and the Global Land Cover Facility (UMD), an ESIP Partner, have developed large data archives with online inventory-level search engines. There is a very large number of scientists that have between 1 and 10 satellite scenes. Most projects lack time and resources to place their holdings online and allow for discovery, search, and download in some manner. SAXTA aims to facilitate projects to make their data available to the community.
Within these communities, there is a need for improved discovery of data collections and search and order of data. Based on peer-to-peer technology, the SAXTA system will extend the utilization of these data sets, which represent government investments of billions of dollars in systems and hardware development and many millions of dollars in data purchases. These purchases include Landsat Pathfinder (Landsat-5), Landsat-7 for regional LCLUC programs and EOS validation, and the Landsat Data buy 1990 and 2000. Widespread user adoption of SAXTA will massively extend the societal benefits of all of these investments.
We believe that, if the cost in time and resources is low, and security is insured, many community members would be willing to federate into a loosely coupled network such as SAXTA. To address this need, Michigan State University and the University of Maryland have partnered with SSAI to develop and deploy such a system. This project is NASA-sponsored cross cutting application that is intended to substantially improve access to high-resolution satellite datasets. The operations concept is that users will simply download free software that will allow them to register their data sets into the system and discover, search, and order datasets for free from other providers within the distributed, loosely coupled, interoperable network.

Empowering a New Approach to Satellite Data Access
Any user can participate in the SAXTA network by downloading, installing and configuring the SAXTA data node.
The user then joins the SAXTA network and is ready to place a search query to the super nodes which will identify the data nodes having data matching the query. The searching data node will then directly contact the relevant provider data nodes and initiate data download. A key feature of the system is that all metadata is visible globally within the network and, once discovered, the data itself can be downloaded directly from provider to user. Since all the data within the SAXTA network are online, the complex process of data ordering and delivery can be eliminated. SAXTA allows for directly jumping from search to free download of data.
The data nodes also provide the user with a data sharing mechanism and allow each user to select precisely the datasets to be shared. From this selection, the SAXTA Software automatically retrieves, extracts and publishes metadata as a collection of records to a set of SAXTA super nodes. Once in the system, the SAXTA software will allow for the maintenance of the metadata records. This includes publishing new metadata data in to the system, as well as un-publishing (removing) old or erroneously registered records. A critical feature of the system is that the data can remain locally hosted and managed. Also, the shared data and metadata are only visible from within the SAXTA community whose access and participation are strictly controlled.

Technology
SAXTA leverages the open source Grid and Peer to Peer JXTA™ [2] technology for providing its core network infrastructure. Sponsored by Sun Microsystems®, Project JXTA provides a standard XML-based set of protocols as well as a JAVA™ reference implementation for the foundations of Peer to Peer and Grid Computing.
JXTA™ was selected because it has the following characteristics:

Generalizes the P2P problem.
Provides for Dynamic Resource Discovery.
Allows Interoperability and ubiquity.
Includes Platform Independence.
Manages network Security, node groups, Stability, Scalability and node communication.
Allows Virtual Network Overlay.

This allows SAXTA to provide data protection by data replication, scalability by allowing user demand to drive replication, and dynamic maintenance of the network – new nodes are constantly appearing while other nodes drop off. The most recent version of JXTA has a scalability target of 1.5 million peers, 300,000 simultaneously connected peers, 1000 elements or more per peer (Landsat images), and 120 peers/minute “churn” rate for peers joining and leaving the network.

Summary
Access to data remains the single greatest barrier to the expanded utilization of Landsat-7 and ASTER data. Widespread user adoption of SAXTA will tear down this barrier. The decentralized architecture of SAXTA will minimizes cost and maximize system scalability and responsiveness. By radically improving access to Landsat and ASTER data, SAXTA will serve as a crosscutting application for the Land Processes and Land Use and Land Cover Change research communities as well as most of the 12 National Applications.

Project Team
The SAXTA software is currently in development. The project is being sponsored by NASA through a grant to Dr. Dave Skole of the Michigan State University Center for Global Change and Landsat.org The SAXTA Principal Investigator is Chris Justice of the University of Maryland Department of Geography. The Lead Software Engineer is Bruno Margerin (SSAI). Dr. Timothy Gubbels (SSAI) serves as the outreach coordinator.

References:
[1]: NASA Open Data Policy: http://globalchange.gov/policies/agency/nasa.html
[2] Project JXTA http://www.jxta.org

My First Blog

2005-04-13T09:04:00.000-07:00

After much encouragement from friends and collegues, I finally decided to start a blog for sharing my experience in Developing with the JXTA (www.jxta.org). JXTA is a set of P2P Protocols. Among others programming languages, this protocols have been implemented in Java. Such implementation is called the "JXTA plateform".

Using JXTA, I am trying to write a "Distributed High Resolution Satellite Data Distribution System". For the rest of us, it is a file sharing system for earth satellite images.

The project's name so far is "SAXTA", but it may change as SAXTA may conflict with the JXTA tm.