Thursday, September 15, 2011

Hosting a Maven Repository on Github for Gradle Builds

Gradle is a great tool for organizing the build logic of software projects. Gradle is able to handle all the dependency management necessary even for very small projects. Under the hood, it relies on Ivy and can deal with Maven repositories.
The question is where to upload my own artifacts which should be accessible as dependencies for other projects. As always, there are several alternatives to choose from. A small excerpt:
  1. Use one of the free Maven repository hosting services as provided by Sonatype for example
  2. Set up an own server with Artifactory or Nexus or something similar (although http, ftp, ssh, ... would suffice, as well)
  3. Create a new Git project and publish it to Github
One benefit of the Sonatype solution is the automated repository synchronization with the Maven Central Repository. However, since the projects have to fulfill a certain set of requirements and I do not need to find my pet projects at Maven Central, this is not my preferred solution.
The company I work for uses Artifactory set up on a dedicated server which is running 24/7 and plays nicely with the continuous integration system. Again, this seems a bit overkill for my private projects.
Since I'm using Git already and have a Github account featuring a very small number of projects, the third alternative seems very appealing to me. This solution requires three basic steps:
  1. Set up and synchronize Git repositories
  2. Upload project artifacts to local Git repository and synchronize it with Github
  3. Configure the Github project as mavenRepo within Gradle build files for depending projects

Step 1: Create a Git Project and Synchronize with Github

For Github project creation I follow the Github Repository Creation Guide. At first I create a repository on Github using the web interface.

Afterwards, I set up a local repository und synchronize it with Github using the following steps:
$ mkdir maven-repository
$ cd maven-repository
$ git init
Initialized empty Git repository in /home/thevis/GIT/maven-repository/.git/
$ touch README.md
$ git add README.md
$ git commit -m 'first commit'
[master (root-commit) eeee5a8] first commit
 0 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 README.md
$ git remote add origin git@github.com:tthevis/maven-repository.git
$ git push -u origin master
Counting objects: 3, done.
Writing objects: 100% (3/3), 209 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To git@github.com:tthevis/maven-repository.git
 * [new branch]      master -> master
Branch master set up to track remote branch master from origin.
$ 
A quick check at Github shows that README.md has actually been pushed to Github and that repository snchronization works fine.

Step 2: Upload Project Artifacts to Local Git Repository

My intention is to establish the following release workflow for my projects: Make and upload a release build using gradle clean build uploadArchives, manually check results in the local Git directory, add it to the Git repository, and finally push it to Github. Well, I know that software releases should not involve manual steps and should be performed on dedicated machines runing build server software and so on and so on...
However, since I'm the only contributor for my projects and the release cycles are somewhat unsettled, this procedure is fairly sufficient for my needs.
Therefore, I change to a project to be released and add following lines to the build.gradle file:
apply plugin: 'maven'

uploadArchives {
	repositories.mavenDeployer {
		repository(url: "file:///home/thevis/GIT/maven-repository/")
	}
}
Executing the release procedure described above yields the following result:
$ gradle clean build uploadArchives
[...suppressed a few output lines here...]
:build
:uploadArchives
Uploading: net/thevis/hadoop/groovy-hadoop/0.2.0/groovy-hadoop-0.2.0.jar to repository remote at file:///home/thevis/GIT/maven-repository/
Transferring 5239K from remote
Uploaded 5239K

BUILD SUCCESSFUL

Total time: 24.443 secs
Just to be sure, I check the result by hand:
$ find /home/thevis/GIT/maven-repository/ -name "*groovy-hadoop*"
/home/thevis/GIT/maven-repository/net/thevis/hadoop/groovy-hadoop
/home/thevis/GIT/maven-repository/net/thevis/hadoop/groovy-hadoop/0.2.0/groovy-hadoop-0.2.0.jar
/home/thevis/GIT/maven-repository/net/thevis/hadoop/groovy-hadoop/0.2.0/groovy-hadoop-0.2.0.jar.md5
/home/thevis/GIT/maven-repository/net/thevis/hadoop/groovy-hadoop/0.2.0/groovy-hadoop-0.2.0.jar.sha1
/home/thevis/GIT/maven-repository/net/thevis/hadoop/groovy-hadoop/0.2.0/groovy-hadoop-0.2.0.pom
/home/thevis/GIT/maven-repository/net/thevis/hadoop/groovy-hadoop/0.2.0/groovy-hadoop-0.2.0.pom.sha1
/home/thevis/GIT/maven-repository/net/thevis/hadoop/groovy-hadoop/0.2.0/groovy-hadoop-0.2.0.pom.md5
Looks good, so I add the files to Git and push them to Github:
$ cd /home/thevis/GIT/maven-repository/
$ git add .
$ git commit -m "released groovy-hadoop-0.2.0"
[master 5026f70] released groovy-hadoop-0.2.0
 9 files changed, 40 insertions(+), 0 deletions(-)
 create mode 100644 net/thevis/hadoop/groovy-hadoop/0.2.0/groovy-hadoop-0.2.0.jar
 create mode 100644 net/thevis/hadoop/groovy-hadoop/0.2.0/groovy-hadoop-0.2.0.jar.md5
 create mode 100644 net/thevis/hadoop/groovy-hadoop/0.2.0/groovy-hadoop-0.2.0.jar.sha1
 create mode 100644 net/thevis/hadoop/groovy-hadoop/0.2.0/groovy-hadoop-0.2.0.pom
 create mode 100644 net/thevis/hadoop/groovy-hadoop/0.2.0/groovy-hadoop-0.2.0.pom.md5
 create mode 100644 net/thevis/hadoop/groovy-hadoop/0.2.0/groovy-hadoop-0.2.0.pom.sha1
 create mode 100644 net/thevis/hadoop/groovy-hadoop/maven-metadata.xml
 create mode 100644 net/thevis/hadoop/groovy-hadoop/maven-metadata.xml.md5
 create mode 100644 net/thevis/hadoop/groovy-hadoop/maven-metadata.xml.sha1
$ git push origin master
Counting objects: 17, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (7/7), done.
Writing objects: 100% (16/16), 5.12 MiB | 98 KiB/s, done.
Total 16 (delta 0), reused 0 (delta 0)
To git@github.com:tthevis/maven-repository.git
   eeee5a8..5026f70  master -> master

Step 3: Use Custom Repository for Depending Projects

There is only one potential pitfall here. The most difficult part is to determine the URL scheme for downloading files from Github. The file groovy-hadoop-0.2.0.jar page reveales the real artifact URL if one hovers the mous pointer over the raw link:
https://github.com/tthevis/maven-repository/raw/master/net/thevis/hadoop/groovy-hadoop/0.2.0/groovy-hadoop-0.2.0.jar. Thus, the maven repository URL to configure is not https://github.com/tthevis/maven-repository but rather https://github.com/tthevis/maven-repository/raw/master/.
With this finding in mind, all the rest is pretty straightforward. For testing purposes I set up the follwing build.gradle:
apply plugin: 'java'
apply plugin: 'eclipse'

version = '0.1.0'
group = 'net.thevis.hadoop'
sourceCompatibility = '1.6'

repositories { 
	mavenRepo urls: 'https://github.com/tthevis/maven-repository/raw/master/'
} 

dependencies {
	compile 'net.thevis.hadoop:groovy-hadoop:0.2.0'
}
And guess what? It works like a charm.

21 comments:

  1. My cousin recommended this blog and she was totally right keep up the fantastic work!

    Maven company Set Up

    ReplyDelete
  2. Can't manage to get it working. Could you please upload your full build.gradle? I've got little experience with this tool.

    ReplyDelete
  3. Best web hosting companies. Reviews, rates, statistics of top hosting companies.
    Find best hosting company at HostingCompaniesz.com

    ReplyDelete
  4. I cant make it work but I´m trying with bitbucket, whats `raw/master`?

    ReplyDelete
  5. Best web hosting and domain reviews and deals in one place!
    domain name

    ReplyDelete
  6. Thanks for the awesome blog!

    ReplyDelete
  7. Offshore Hosting By #1 Trusted Offshore Host Provider. http://offshorededicated.net/

    ReplyDelete
  8. OffshoreDedi offers high quality DMCA Ignored Hosting. We make offshore hosting simple for you to use with reliable servers and one-click installers. Your data is safe with us as it’s is kept in a high security facility in an offshore jurisdiction assuring your privacy.

    ReplyDelete
  9. https://webcare360.com Provides you safe and secure cheap Offshore Hosting Best offshore hosting and DMCA ignore Hosting with 99.9% Up time Guarantee, with Ddos protection.

    ReplyDelete
  10. Get Fully DMCA Ignored Hosting with 99.9% uptime Guarantee in Cheapest Prices. All offshore servers are hosted in multiple locations so you can run your websites very smoothly without worrying about DMCA complaints. We are offering offshore shared hosting, Offshore VPS & Offshore Dedicated Servers

    ReplyDelete
  11. This comment has been removed by the author.

    ReplyDelete
  12. you would not really want to deal with those rip-off travel agents”
    webcare360

    ReplyDelete
  13. Many blogs like this cover subjects that just aren’t covered by magazines.
    offshoreservers.net

    ReplyDelete
  14. StreamingDedi Offers DMCA Friendly Offshore Streaming Servers. We don’t require any information from you expect a valid email.

    ReplyDelete