JMeter SFTP Request

In my previous post, we have seen how to utilize an external groovy script to make some FTP Requests.

Now there is slight change in the requirement. FTP got changed to SFTP, which means it’s more secure.

We can’t use our FTP Samplers to create a SFTP request to authenticate using SSH.

Originally I built the SSH FTP Sampler using the instructions in HOW TO BUILD SSH SFTP SAMPLER FOR JMETER? (BY IMPLEMENTS TESTBEAN)

We have to check out the source code from https://github.com/yciabaud/jmeter-ssh-sampler and then generating the target by building it using maven.

Also we need Jsch. JSch is a java implementation of SSH. This allows us to connect to an FTP server for file transfer. http://sourceforge.net/projects/jsch/files/jsch.jar/0.1.53/jsch-0.1.53.jar/download

But just to make this process simpler, I checked-in the two jar files required for creating SSH FTP request here – https://github.com/linkeshkanna/Jmeter.SSH.FTP.Request

This contains two jar files.

  1. jmeter-ssh-sampler-1.0.2-SNAPSHOT.jar
  2. jsch-0.1.53.jar

To install this in Jmeter,

  1. Copy the jmeter-ssh-sampler-1.0.2-SNAPSHOT.jar to the “Jmeter/Lib/ext” directory.
  2. Copy the “jsch-0.1.53.jar” to the “Jmeter/Lib” directory
  3. Restart Jmeter.

I have also added a sample test to list of the directory contents in a Public SFTP Server – https://github.com/linkeshkanna/Jmeter.SSH.FTP.Request/blob/master/SSH-FTP.jmx

Open this JMX file in your JMeter and run it to see this in action.

Jmeter Groovy Scripting – Call External Scripts

In my previous post, Jmeter BeanShell Scripting – Call External Scripts we have discussed how to user Jmeter Bean Shell script to call external .bsh scripts.

Now in this post, we are going to see how to do the same functionality with JSR 223 sampler + Groovy scripting.

It’s always recommended to use JSR 223 Sampler instead of Bean shell Sampler. I recently got a nice comparison between these two in this post – BeanShell Vs JSR223 Sampler

The JSR223 test elements have a compilation feature that can significantly increase performance.

To get benefit from this feature, Use external .groovy Script files instead of in lining them. This will make Jmeter compile them if this feature is available on Script Engine and cache them.

To get additional benefits from Caching and compilation, language engine used for scripting must implement JSR223 compilable interface.

Groovy is one of the JSR223 compilable interface. But Java, Bean Shell and Java script are not.

Since JMeter 2.8, JSR223 Test Elements using external Script file are now pre compiled and this enables great performance enhancements.

So we are going to use JSR223 test elements with external Groovy script files.

Pre Requisites:

Groovy isn’t shipped with Jmeter. So it needs to be downloaded separately.  To get started:

  1. Download latest groovy binary bundle from http://groovy-lang.org/download.html
  2. Copy all the Jar files under “groovy-{versionNumber}\lib” and paste it into “Jmeter/lib/ext” folder of your Jmeter installation directory.
  3. Restart Jmeter.

Scenario:

Now we are going to do the same scenario as in the previous post. But instead of http requests, I am gonna do FTP Requests. In short following is the scenario.

I have to do upload some flat files to a FTP server through FTP requests and the flat files are available on a sub directory where my test plan file – .JMX file exists. So I have to parse the list of available files and have to pass the file names as input to my FTP Requests.

I am going to create two external groovy script files for this scenario.

One is for listing out the files under sub directory and putting the absolute path of the Flat files it into a Jmeter variable.

Another script is for extracting the file names alone from the absolute path Jmeter variables.

Following is the “FlatFilePathProcessor.groovy” script file content and I have saved this file where our test plan file exists.


import org.apache.commons.io.FilenameUtils;
import org.apache.jmeter.services.FileServer;

String subDirectory = args[0];
String testPlanFile = FileServer.getFileServer().getBaseDir().replace('\\', '/');
//String testPlanFileDir = FilenameUtils.getFullPathNoEndSeparator(testPlanFile).replace('\\', '/');

vars.put("testPlanFileDir", testPlanFile);

File folder = new File(testPlanFile + "/" + subDirectory + "/");
File[] listOfFiles = folder.listFiles();
int counter = 1;
for (File file : listOfFiles) {
if (file.isFile()) {
vars.put(subDirectory + "_" + counter, testPlanFile + "/" + subDirectory + "/" + file.getName());
vars.put("FileNames" + "_" + counter, file.getName());
}
counter++;
}


Following is the “ExtractFileName.groovy” script file content and I have also saved this file where our test plan file exists.


String fullPath = vars.get("FlatFiles");
int index = fullPath.lastIndexOf("/");
String fileName = fullPath.substring(index + 1);
vars.put("FileName", fileName);

Following is the project structure that we are going to create.

1

Create a test plan and then create a thread group below that.

First we are going to add JSR 223 sampler in our thread group. – the job of this sampler is to call the FlatFilePathProcessor.groovy file with sub directory name as a parameter. So the request will list out the flat files under sub directory and putting the absolute path of each and every flat file it into a Jmeter variable. Following is the request.

2

Next we are going to create a ForEach Controller, which will loop through the FTP for each flat file variable created in the above step. Following is our controller.

3

Now we are going to create JSR 223 Pre-processor. The job of this test element is to call ExtractFileName.groovy to extract the file name from the absolute path. Because for FTP request we need both the file name as well as the absolute file path.

Also one more thing I have noticed is, in FlatFilePathProcessor.groovy itself, we have created two variables one contains the absolute path and another for the file name. But now we are under the ForEach controller, which will process/loop through only one variable. I have not seen any reference where the ForEach controller loop through multiple variables. Following is the request.

4

Here comes the actual FTP Request, with simple response code assertion. We have used two variables here – one for filename that we got from preprocessor [ExtractFileName.groovy] and another for the absolute file path of the flat file which we are going to upload, that we got from sampler [ExtractFileName.groovy].

5

Add debug sampler and your favorite listener to see this in action.

The source code of the sample program is here – https://github.com/linkeshkanna/Jmeter.External.Groovy.Scripts

Jmeter BeanShell Scripting – Call External Scripts

Recently I come across a requirement to use Beanshell Scripting in Jmeter. So following is my requirement.

  1. Create 2 thread groups for 2 Http Requests to make API Calls and in that http request, my Post body should be iterated for around 9000 flat files in two different directories. Following is the Project Structure I have.1
  1. For HttpRequest1 under ThreadGroup1 I have to iterate for all the text files in ‘DirectoryOne’
  2. For HttpRequest2 under ThreadGroup2 I have to iterate for all the text files in ‘DirectoryTwo’

Challenges:

  1. Need to use absolute Path for pointing to the directory where my files exist – Not a better solution, when running from the CI build, this absolute path will vary.
  2. Need to use another CSV file(s) to configure the list of filenames available under ‘DirectoryOne’ and ‘DirectoryTwo’ – Not a feasible solution. If there is any new addition of new files under any directory, I have to update my CSV file to include the new file names as well.
  3. Read the text file content and pass it as a post body request.

Let’s start solving this issue.

I started exploring the bean shell sampler in Jmeter. I have created a bean shell script to process the flat files under ‘DirectoryOne’. Following is the bean shell script


import org.apache.jmeter.gui.GuiPackage;

import org.apache.commons.io.FilenameUtils;



String testPlanFile = GuiPackage.getInstance().getTestPlanFile();

String testPlanFileDir = FilenameUtils.getFullPathNoEndSeparator(testPlanFile).replace('\\', '/');



vars.put("testPlanFileDir", testPlanFileDir);



File folder = new File(testPlanFileDir + "/" + "DirectoryOne" + "/");

File[] listOfFiles = folder.listFiles();

int counter = 1;

for (File file : listOfFiles) {

if (file.isFile()) {

vars.put("DirectoryOne" + "_" + counter, testPlanFileDir + "/" + "DirectoryOne" + "/" + file.getName());

 }

counter++;

}

2

If you look into the above issue, I have pointed to the root directory [whatever it may be] where my test plan file exist and then I am appending my sub directory name to it.

But for ThreadGroup2, I have to copy paste the same bean shell sampler with same script except for ‘DirectoryTwo’ instead of ‘DirectoryOne’. So I thought of reusing the same bean shell script for all the thread groups. The only option is to externalize it.

Now I have created a file called “FilePathProcessor.bsh” inside the same directory where my test plan file exists. One interesting thing that I have noticed in the bean shell sampler was it will even run the external .bsh scripts with arguments passed from Jmeter.

Again I have to use a relative path to locate this external .bsh file. Following is the small snippet that you can use to point your bean shell Sampler – “Script File Text Box”.


${__BeanShell(import org.apache.jmeter.services.FileServer; FileServer.getFileServer().getBaseDir();)}${__BeanShell(File.separator,)}FilePathProcessor.bsh

This means, no need for providing the absolute path to locate the external bean shell script. The above code will point to the base directory where our test plan/.JMX file exists.

Now we have to pass the sub directory names as an argument in the “Parameters (-> String Parameters and String []bsh.args)” text box.

For first ThreadGroup, I have passed it as ‘DirectoryOne’.

For Second ThreadGroup, I have passed it as ‘DirectoryTwo’.

So my updated external bean shell script will look like,


import org.apache.jmeter.gui.GuiPackage;

import org.apache.commons.io.FilenameUtils;



String subDirectory = bsh.args[0];

String testPlanFile = GuiPackage.getInstance().getTestPlanFile();

String testPlanFileDir = FilenameUtils.getFullPathNoEndSeparator(testPlanFile).replace('\\', '/');



vars.put("testPlanFileDir", testPlanFileDir);



File folder = new File(testPlanFileDir + "/" + subDirectory + "/");

File[] listOfFiles = folder.listFiles();

int counter = 1;

for (File file : listOfFiles) {

if (file.isFile()) {

vars.put(subDirectory + "_" + counter, testPlanFileDir + "/" + subDirectory + "/" + file.getName());

}

counter++;

}

2.1

So my final project structure will look like,

3

FilePathProcessor – the Bean Shell Sampler. This will produce JmeterVariables for each file located in the sub directory. For example, when iterating over the files in ‘DirectoryOne’ and if it has two files exists, this will produce two Jmeter Variables with Prefix ‘DirecotryOne’. You can see this in action using Jmeter Debug Sampler.

DirectoryOne_1 = “D:\DirectoryOne\Test1.txt”

DirectoryOne_2 = “D:\DirectoryOne\Test2.txt”.

I have created ‘ForeEach Controller’ to parse these Jmeter Variables.

4

The “Output Variable Name” in the ‘ForEach Controller’ will contain the absolute path of the text files. Now to parse the text inside a text file, we can simply use in-built Jmeter functions. Go-to Body Data of HttpRequest and write ${__FileToString(${DirectoryOneFileNames})}

The _FileToString in built Jmeter function will return the file content of a given flat file.

5

So that’s it. Now we have solved all the possible challenges in-front of us. If you want to have complete information about this project, you can simply have a look at this sample script here.

https://github.com/linkeshkanna/Jmeter.External.BeanShell.Scripts