The Techjoomla Blog
Packaging system for Joomla extensions
Joomla extension package is a combination of core, sub-extension, libraries, layouts, core folder, etc.
Many of Techjoomla’s extensions are shipped with other Techjoomla extensions and infrastructure extensions. For eg:-JTicketing is shipped with JLike along with TJ Vendor, TJ Notifications and TJ Reports. Let's assume that each extension code is maintained in separate GitHub repositories.
Now, when it comes to package building it's a tedious task to remember the branch name of each extension, acquire the latest code from different repositories and then build the package manually.
No more worries as we have written a script which will make your life easier.
Here are some prerequisites you need to know before creating the package.
Prerequisites
- One should have knowledge of
- Jenkins CI/CD
- Jenkins pipeline script and
- GitHub.
Let’s look at the example of Techjoomla JGive(It’s our own crowdfunding extension) package.
JGive package contains
- Layouts
- Libraries
- Modules
- Plugins
- Extensions
- com_jgive.zip
- com_activitystream.zip etc.
Here libraries, plugins, and sub extensions have their own GitHub repository.
A. package.json:
Create package.json file, this file will tell the script source of extension, libraries, plugins, etc and destination in the package.
Path: your_github_repo/build/package.json
{
"package_name": "name_of_the_package",
"core_files": {
"src": "source of the package core files",
"dest": "builds/(No change here)"
},
"subextensions": [ // List of subextensons: Components, libraries, plugins etc.
{
"name": "Github repository name",
"repoUrl": "Github repository clone url",
"src": "Source code folder path",
"dest": "builds/destination folder path",
"zip_name": "optional: if sub extension needs to be compressed",
"format": "zip/folder ('zip': if file to be compressed otherwise use 'folder')"
"include/exclude":[
"Folder to include/exclude"
]
}
],
"versions": {
"1.0.0 (Package version number)": {
"Github repository name(must be exactly same used above in subextensions)": {
"version": "Github Tag",
"branch": "Github branch name"
}
}
}
}
package_name: Name of the package
core_files: The core files. E.g layout, modules that are maintained in the same repository of the core component.
sub extensions: List of extensions contained in the package.
versions: This gives us the flexibility to build the package for any version. Version object should have version property for each version and extension branch.
E.g
{
"package_name": "pkg_jgive",
"core_files": {
"src": "./jgive/src/*",
"dest": "builds/"
},
"subextensions": [
{
"name": "jgive",
"repoUrl": "git@github.com:techjoomla/jgive.git",
"src": "./jgive/src/com_jgive/*",
"dest": "builds/com_jgive",
"zip_name": "com_jgive",
"format": "zip"
},
{
"name": "plg_tjassetsloader",
"repoUrl": "git@github.com:techjoomla/plg_tjassetsloader.git",
"src": "./plg_tjassetsloader/src/*",
"dest": "builds/plugins/system/tjassetsloader",
"format": "folder"
},
{
"name": "joomla-payments",
"repoUrl": "git@github.com:techjoomla/joomla-payments.git",
"src": "joomla-payments/code/plugins/",
"dest": "builds/plugins/payment/",
"format": "folder",
"include": [
"2checkout",
"alphauserpoints",
"authorizenet",
"bycheck",
"byorder",
"jomsocialpoints",
"paypal"
]
},
{
"name": "com_activitystream",
"repoUrl": "git@github.com:techjoomla/com_activitystream.git",
"src": "./com_activitystream/src/*",
"dest": "builds/packages/com_activitystream",
"zip_name": "com_activitystream",
"format": "zip"
}
],
"versions": {
"2.3.0": {
"jgive": {
"branch": "release-2.3.0"
},
"plg_tjassetsloader": {
"branch": "master"
},
"joomla-payments": {
"branch": "release-1.0"
},
"com_activitystream": {
"branch": “release-3.0"
}
}
}
B. package.groovy
Create a package.groovy file, just update the version value whichever you want to build and paste the code as it is.
path: your_github_repo/build/pipelines/package.groovy
//!/usr/bin/env groovy
// Get / Set release name
def version = '2.3.0'
echo version
pipeline {
agent any
stages {
stage('Cleanup') {
steps {
script {
// Cleanup previous stuff
sh("rm -rf scm")
sh("rm -rf builds")
// Cleanup jlike git folder, files
sh("rm -rf .git")
sh("rm -rf .gitlab/merge_request_templates")
sh("rm -rf build")
// Make directories needed to generate build
sh("mkdir builds")
sh("mkdir scm")
}
}
}
stage('Checkout') {
steps {
script {
// This is important, we need clone into different folder here,
// Because, as part of tag based pull, we will be cloning same repo again
dir('scm') {
checkout scm
}
}
}
}
stage('Cleanup-repos') {
steps {
script {
def props = readJSON file: 'scm/build/package.json'
props['subextensions'].eachWithIndex { item, index ->
// cleaup subextensions
sh("rm -rf $item.name")
}
}
}
}
stage('Init') {
steps {
script {
def props = readJSON file: 'scm/build/package.json'
// Do clone all subextensions repos by checking out corresponding release branch
props['subextensions'].eachWithIndex { item, index ->
sh("git clone --branch " + props['versions'][version][item.name]["branch"] + " --depth 1 $item.repoUrl")
}
}
}
}
stage('Copy files & Make zip of subextension') {
steps {
script {
def props = readJSON file: 'scm/build/package.json'
// Copy core files
sh("cp -r " + props["core_files"].src + " " + props['core_files'].dest)
// Copy Make the zips
props['subextensions'].eachWithIndex { item, index ->
sh("mkdir -p " + item.dest)
if (item.include && item.exclude)
{
error "Use either include or exclude. Both together not supported!."
}
def simple_copy = 0
if (item.include)
{
def count = item.include.size()
if (count > 0)
{
def include = item.include.collect { "$item.src/$it" }.join(' ')
sh("cp -r $include $item.dest")
}
else
{
simple_copy = 1
}
}
else if(item.exclude)
{
def count = item.exclude.size()
if (count > 0)
{
def exclude = item.exclude.collect { "--exclude $it" }.join(' ')
sh("rsync -avz $exclude $item.src $item.dest")
}
else
{
simple_copy = 1
}
}
else
{
simple_copy = 1
}
if (simple_copy == 1)
{
sh("cp -r $item.src $item.dest")
}
// Create subextension zip and remove folder which is not needed after zip
if (item.format == "zip")
{
sh("cd $item.dest && zip -rq ../$item.zip_name" + ".zip .")
sh("rm -rf $item.dest")
}
}
}
}
}
stage('Build Package') {
steps {
script {
// // Get commit id
// // @TODO - needs to define shortGitCommit at global level
def gitCommit = ''
def shortGitCommit = ''
def props = readJSON file: 'scm/build/package.json'
// // For branch based build - we need the revision number of tag checked out,
// Custom DIR
dir('scm') {
gitCommit = sh(returnStdout: true, script: 'git rev-parse HEAD').trim().take(8)
shortGitCommit = gitCommit[0..7]
echo gitCommit
echo shortGitCommit
}
// Now we are good to create zip for component
sh('cd builds && zip -rq ../' + props['package_name'] + '_v' + version + "_" + shortGitCommit + '.zip .')
archiveArtifacts props['package_name'] + '_v' + version + "_" + shortGitCommit + '.zip'
}
}
}
stage('Cleanup folders') {
steps {
script {
// Cleanup, so next time we get fresh files
sh("rm -r builds")
}
}
}
}
}
C. Jenkins Job
Setup the Jenkins job to run the pipeline script, set the script path as build/pipelines/package.groovy, run the job and your package will be ready within a couple of minutes:-).
Do let us know if you are using anything cool to build the package.
When you subscribe to the blog, we will send you an e-mail when there are new updates on the site so you wouldn't miss them.
About the author
Blog Topics
