Terse, unterse and transfer datasets between z/OS and other platforms via FTP

Tersing files can be compared to tarring, zipping, rarring files on the x86 platform. It allows you to store a file or multiple files (members) in an archive. The archive is easily transferable and when the data is unpacked, it is guaranteed to be the same as the original source. Especially when transferring PDS’es and variable blocked (VB, VBS) datasets, this is important. Transferring normal datasets over FTP through other platforms from z/OS can be challenging an this article should provide some help.

A common way to transfer data from MVS/System z to another flatform is by using FTP. And often this causes problems regarding line breaks, special characters, preallocation of datasets when uploading,…
A tersed archive can be safely transferred with FTP and still, after extracting the data on it’s target, the data is integer???. In case there would be some problem with the tersed archive (it can always get corrupted in some way), terse won’t be able to unpack your data, which means you know that the data is faulty rather that getting some, incorrect, data.

Since z/OS 1.9, the terse utility has been “redesigned” and is now called AMATERSE. Fortuantely, there is an alias for AMATERSE to the classic name TRSMAIN.
In the examples, I’ll use TRSMAIN since that means that the example should also work on z/OS < 1.9. The examples were done on z/OS 1.10.

Tersing is limited to sequential datasets, both fixed block and variable blocked and normal PDS. PDSE and VSAM are unfortunately not supported. To terse a VSAM, you can use a workaround which maybe, when I find some more time, describe later.

Terse (pack) a sequential dataset

To terse a sequential dataset, fixed block or variable blocked, we need the following JCL:

//TERSEQ   JOB (,),REGION=0M,NOTIFY=&SYSUID,CLASS=A    
//****************************************************                  
//TERSEQ   EXEC PGM=TRSMAIN,PARM=PACK                                   
//SYSPRINT DD SYSOUT=*                                                  
//INFILE   DD DSN=IBMUSER.DATA.SEQ,
//            DISP=SHR                             
//OUTFILE  DD DSN=IBMUSER.DATA.SEQ.TERSED,                              
//            DISP=(NEW,CATLG),                                         
//            SPACE=(CYL,(1,1))

line 1 is the jobcard which probably needs to be adjusted to your environment
line 3 calls the program TRSMAIN which, as mentioned earlier, can point to AMATERSE. Parameter is “PACK”
line 5-6 contains the DD-statement for the dataset that will be tersed, disposition share since we’ll only read the dataset
line 7-9 contains the DD-statement for the output, so the “archive”, disposition NEW,CATLG since we’ll allocate it as a new dataset and catalog it. As for the DCB, there is no need to add one since the terse utility overrides it anyway if it’s not correct.

Space allocation for the output heavily depends on the type of data that you are tersing so there is no golden rule in this. One approach would be to add a release (RLSE) parameter on the space allocation.

IBMUSER.DATA.SEQ is a normal, fixed block, dataset containing some dummy data for this example. It was allocated as follows:

                              Data Set Information                              
 Command ===>                                                                   
                                                                                
 Data Set Name . . . . : IBMUSER.DATA.SEQ                                       
                                                                                
 General Data                           Current Allocation                      
  Management class . . : **None**        Allocated cylinders : 1                
  Storage class  . . . : **None**        Allocated extents . : 1                
   Volume serial . . . : ZASYS1                                                 
   Device type . . . . : 3390                                                   
  Data class . . . . . : **None**       Current Utilization                     
   Organization  . . . : PS              Used cylinders  . . : 1                
   Record format . . . : FB              Used extents  . . . : 1                
   Record length . . . : 80                                                     
   Block size  . . . . : 27920                                                  
   1st extent cylinders: 1                                                      
   Secondary cylinders : 1                                                      
   Data set name type  :                 SMS Compressible  :   NO               
                                                                                
   Creation date . . . : 2014/10/08      Referenced date . . : 2014/10/08       
   Expiration date . . : ***None***

After submitting the job TERSEQ, you should get something like this in the output of the job (in SDSF or Beta92)

** AMA572I STARTING TERSE ENCODE   PACK         05:07:07  10/08/2014 ****       
** AMA527I  INPUT  - DDNAME : INFILE   DSNAME: IBMUSER.DATA.SEQ                 
** AMA528I  OUTPUT - DDNAME : OUTFILE  DSNAME: IBMUSER.DATA.SEQ.TERSED          
** AMA573I TERSE COMPLETE ENCODE   PACK         05:07:08  10/08/2014 ****       
** AMA504I  RETURN CODE: 0

As you can see, our job ended with RC=0 and if all went well, we should see a newly allocated dataset, the archive containing the tersed data:

                              Data Set Information                              
 Command ===>                                                                   
                                                                                
 Data Set Name . . . . : IBMUSER.DATA.SEQ.TERSED                                
                                                                                
 General Data                           Current Allocation                      
  Management class . . : **None**        Allocated cylinders : 1                
  Storage class  . . . : **None**        Allocated extents . : 1                
   Volume serial . . . : ZASYS1                                                 
   Device type . . . . : 3390                                                   
  Data class . . . . . : **None**       Current Utilization                     
   Organization  . . . : PS              Used cylinders  . . : 1                
   Record format . . . : FB              Used extents  . . . : 1                
   Record length . . . : 1024                                                   
   Block size  . . . . : 6144                                                   
   1st extent cylinders: 1                                                      
   Secondary cylinders : 1                                                      
   Data set name type  :                 SMS Compressible  :   NO               
                                                                                
   Creation date . . . : 2014/10/08      Referenced date . . : 2014/10/08       
   Expiration date . . : ***None***

The dataset is a PS, FB with a record length of 1024. This is always the case for a tersed file and important to know when uploading a tersed file via FTP to another z/OS system.

Unterse (unpack) a sequential dataset

Now that we know what to do to terse a certain dataset, it’s good to know how to get our original data back out of there :)
To do so, we need to create a JCL which is very similar to the one used to pack:

//UNTERSEQ JOB (,),REGION=0M,NOTIFY=&SYSUID,CLASS=A
//****************************************************                  
//UNTERSEQ EXEC PGM=TRSMAIN,PARM=UNPACK                                 
//SYSPRINT DD SYSOUT=*                                                  
//INFILE   DD DSN=IBMUSER.DATA.SEQ.TERSED,
//            DISP=SHR                      
//OUTFILE  DD DSN=IBMUSER.DATA.SEQ.UNTERSED,                            
//            DISP=(NEW,CATLG),                                         
//            SPACE=(CYL,(1,1))

This time, we specify parameter “UNPACK” on the 3rd line of the JCL, telling TRSMAIN, or AMATERSE, that we want to unpack the archive.
line 5-6 contains the input, which is the output of our terse (pack) operation. This can be either on the same system or after transferring the dataset to another system (see further).
line 7-9 contains the output of the job and should return us the original input dataset. As with the terse job, there is no need to give a DCB since the terse utility will allocate the dataset in the same way as the original input.
regarding space, it’s good to know the original size of the input. If you don’t, you can take 2-3 times the size of the tersed file to be sure that there won’t be any issues regarding space allocation. You can always specify the RLSE parameter to release the extranious space afterwards.

After submitting the job, it should produce something like this:

** AMA572I STARTING TERSE DECODE   UNPACK       05:08:06  10/08/2014 ****       
** AMA527I  INPUT  - DDNAME : INFILE   DSNAME: IBMUSER.DATA.SEQ.TERSED          
** AMA528I  OUTPUT - DDNAME : OUTFILE  DSNAME: IBMUSER.DATA.SEQ.UNTERSED        
** AMA555I  THE VALUES ARE:  BLKSIZE= 27920   LRECL=80      PACKTYPE=PACK    REC
** AMA573I TERSE COMPLETE DECODE   UNPACK       05:08:07  10/08/2014 ****       
** AMA504I  RETURN CODE: 0

In the output of the job you can see that the terse utility allocates the dataset with the correct DCB, equal to the original input.
The only thing that you need to know, as mentioned above, is a rought idea about the unpacked size.

To be sure that the output was allocated correctly:

                              Data Set Information                              
 Command ===>                                                                   
                                                                                
 Data Set Name . . . . : IBMUSER.DATA.SEQ.UNTERSED                              
                                                                                
 General Data                           Current Allocation                      
  Management class . . : **None**        Allocated cylinders : 1                
  Storage class  . . . : **None**        Allocated extents . : 1                
   Volume serial . . . : ZASYS1                                                 
   Device type . . . . : 3390                                                   
  Data class . . . . . : **None**       Current Utilization                     
   Organization  . . . : PS              Used cylinders  . . : 1                
   Record format . . . : FB              Used extents  . . . : 1                
   Record length . . . : 80                                                     
   Block size  . . . . : 27920                                                  
   1st extent cylinders: 1                                                      
   Secondary cylinders : 1                                                      
   Data set name type  :                 SMS Compressible  :   NO               
                                                                                
   Creation date . . . : 2014/10/08      Referenced date . . : 2014/10/08       
   Expiration date . . : ***None***

The contents of the dataset should be equal to the original contents of IBMUSER.DATA.SEQ

Terse (pack) a PDS

To pack a PDS is exactly the same as packing a “normal” sequential dataset so I’ll not go into detail and only show the JCL used to pack a PDS.

//TERSEPDS JOB (,),REGION=0M,NOTIFY=&SYSUID,CLASS=A                             
//****************************************************                  
//TERSEPDS EXEC PGM=TRSMAIN,PARM=PACK                                   
//SYSPRINT DD SYSOUT=*                                                  
//INFILE   DD DSN=IBMUSER.DATA.PDS,
//            DISP=SHR                             
//OUTFILE  DD DSN=IBMUSER.DATA.PDS.TERSED,                              
//            DISP=(NEW,CATLG),                                         
//            SPACE=(CYL,(1,1))

Unterse (unpack) a PDS

To unpack a PDS is, as with packing, very similar to unpacking a sequential dataset. The only difference is that you need to be carefull to allocate a PDS for the output and specify a directory size on the space parameter:

//UNTERPDS JOB (,),REGION=0M,NOTIFY=&SYSUID,CLASS=A
//****************************************************                  
//UNTERPDS EXEC PGM=TRSMAIN,PARM=UNPACK                                 
//SYSPRINT DD SYSOUT=*                                                  
//INFILE   DD DSN=IBMUSER.DATA.PDS.TERSED,
//            DISP=SHR                      
//OUTFILE  DD DSN=IBMUSER.DATA.PDS.UNTERSED,                            
//            DISP=(NEW,CATLG),                                         
//            SPACE=(CYL,(1,1,1))

The only difference with the previous unpack job are the names of the datasets and the extra parameter to allocate directory block on line 9.

The output of the job doesn’t show big differences as well:

** AMA572I STARTING TERSE DECODE   UNPACK       05:11:20  10/08/2014 ****       
** AMA527I  INPUT  - DDNAME : INFILE   DSNAME: IBMUSER.DATA.PDS.TERSED          
** AMA528I  OUTPUT - DDNAME : OUTFILE  DSNAME: IBMUSER.DATA.PDS.UNTERSED        
** AMA555I  THE VALUES ARE:  BLKSIZE= 27920   LRECL=80      PACKTYPE=PACK    REC
** AMA573I TERSE COMPLETE DECODE   UNPACK       05:11:23  10/08/2014 ****       
** AMA504I  RETURN CODE: 0

By looking at the allocated output dataset, we can see that we have a PDS and the contents is the same as in the original IBMUSER.DATA.PDS:

                              Data Set Information                              
 Command ===>                                                                   
                                                                                
 Data Set Name  . . . : IBMUSER.DATA.PDS.UNTERSED                               
                                                                                
 General Data                          Current Allocation                       
  Volume serial . . . : ZASYS1          Allocated cylinders : 2                 
  Device type . . . . : 3390            Allocated extents . : 2                 
  Organization  . . . : PO              Maximum dir. blocks : 1                 
  Record format . . . : FB                                                      
  Record length . . . : 80                                                      
  Block size  . . . . : 27920          Current Utilization                      
  1st extent cylinders: 1               Used cylinders  . . : 2                 
  Secondary cylinders : 1               Used extents  . . . : 2                 
                                        Used dir. blocks  . : 1                 
  Creation date . . . : 2014/10/08      Number of members . : 3                 
  Referenced date . . : 2014/10/08                                              
  Expiration date . . : ***None***

Safely transfer tersed datasets over FTP between different platforms

To transfer you previously tersed dataset from z/OS (or z/VM) to another platform, usually there is nothing special to do.
You can use a graphical FTP client like Filezilla or use a command-line untility. Since there is no preallocation of files and a file is a file, we only need to pay attention that the transfer is done binary and not in ascii.

To transfer a tersed dataset back to z/OS or to another z/OS-host, you need to be more careful since datasets are getting allocated by FTP.
The FTP daemon running on z/OS contains some rules about how to allocate uploaded files and unless you defined a special convention for tersed datasets, you will need to specify the allocation parameters yourself when uploading a tersed dataset.

Tersed datasets need to be uploaded in the following format:
– lrecl: 1024
– refcm: FB
– blocksize: 1024

These allocation parameters can be communicated to the host, with the FTP-command quote site.

Besides the DCB-parameters, it’s also handy to specify the allocated space for the new dataset that will be uploaded. That can be done with the following parameters

– primary
– secondary

A small example uploading a tersed dataset to a z/OS host:

D:data\>ftp -i mvshost.test.local
Connected to mvshost.test.local.
220-FTPD1 IBM FTP CS V1R10 at LOOPBACK, 16:24:18 on 2014-10-08
220 Connection will close if idle for more than 5 minutes.
Username: (mvshost.test.local:(none)): IBMUSER
331 Send password please.
Password:
230 IBMUSER is logged on. Working directory is "IBMUSER.".
ftp> quote site primary=1
200 SITE command was accepted
ftp> quote site secondary=1
200 SITE command was accepted
ftp> quote site lrecl=1024
200 SITE command was accepted
ftp> quote site blocksize=1024
200-BLOCKSIZE must be at least 4 more than LRECL for RECFM VB
200-BLOCKSIZE being set to 1028
200 SITE command was accepted
ftp> quote site recfm=FB
200-BLOCKSIZE must be a multiple of LRECL for RECFM FB
200-BLOCKSIZE being set to 1024
200 SITE command was accepted
ftp> bin
200 Representation type is Image
ftp> put data.tersed IBMUSER.UPLOAD.TERSED
200 Port request OK.
125 Storing data set IBMUSER.UPLOAD.TERSED
250 Transfer completed successfully.
ftp: 616 bytes sent in 0,63 seconds 106,49 kB/s.
ftp> quit

You can see the site-commands regarding space on line 9 and 11 and the site-commands regarding the DCB on line 13,15 and 19.
line 23 tells that we want the transfer to be binary (don’t forget this one!)

Using the above example, assures that the terse file gets uploaded correctly and gives you some more flexibility transferring sequential datasets or PDS through other platforms.

12 thoughts on “Terse, unterse and transfer datasets between z/OS and other platforms via FTP

  1. I am actually pleased to read this website posts which consists of lots off useful
    data, thanks for providing these kinds of information.

  2. Hi Jens,
    have you made already a TERSE for an OMVS (HFS) file?
    Does it function at all?

    Greets,
    Gábor

    • Hi,

      Theoretically, this shouldn’t make any difference but I no longer have access to a mainfraime or z/OS setup, Sorry :)

      • Hi Jens,

        thanks. Now I’ve got a problem with the PC Terse. When I transmit an – on the mainframe tersed – textfile, it could be untersed on the PC side wit no problem. The same process with a tersed SMF recordfile results in a binary file on the PC side with a header which could not to be interpreted on the PC side Terse…

  3. Hi,
    I am trying to use terse as a replacement product for PKZIP. I have successfully packed and unpack the dataset via TERSE on the mainframe specifing the output dataset as a PDS. I transferred the file down to the PC and executed the PC Version of terse. Is there anyway that PC Terse will create multiple files ? Aren’t there header files within the compressed file. Any help is greatly appreciated.

  4. Hello Gooch: I am trying to do the same as you. Do you have any info. If I send a tersed file (tersed by IBM program TRSMAIN ) from Mainframe to Window and Linux, what program I can use in Window or Linux to untersed this file. Please let me know. Thank you.

    • Hi THANH N PHAM,

      Did you get any answer for the above? How did you do it in UNIX side? I am also looking for a similar solution.

      Thanks
      Vinod

Leave a Reply

Your email address will not be published. Required fields are marked *