FAQ

Most Popular

The .cpb file describes the search capabilities of a source. The .pmf is the pre-mapping file that allows administrators to map search/index attributes that apply before the Muse ISR stylesheet is used to format the search for the source (thus “Pre”-mapping). The .cpb file lists all the search indexes and Boolean operators which are supported by the Source (whatever the actual, native syntax of the attribute). It is also a place where other source capabilities could be added later for easy import into the Muse InfoBase. The search limiter sets are one example. The .pmf file provides a mapping from one attribute in ISR to one attribute in translator. The .pmf mapping is TO an attribute which is handled into the ISR translator. All of the attributes handled (supported) in the ISR translator have been automatically extracted and recorded into the appropriate section in .cpb file. By default, the PMF files map unsupported attributes to keyword.

MuseGlobal provides the following method for search and retrieval of EBSCOhost content: EBSCO Integration Toolkit (EIT)

This is a SOAP-based Web Service approach which provides the optimal combination of performance and completeness. You can find the EIT Source Packages by searching in your Muse Console’s “Add Sources” section in the “IDs Containing” field for the term EIT. EIT SPs have Source IDs which start with the string EBSCOEIT.

URLs:

The HTTP Source Packages for EBSCO search on the URL
http://search.ebscohost.com/. The EIT Source Packages use
http://eit.ebscohost.com/Services/SearchService.asmx as the Home URL and Search URL.

AUTHENTICATION:
Authentication for the EBSCOEIT Source Packages is two-fold.

1) Authentication for search and retrieval can done by user/pw (with the USER_NAME and USER_PASSWORD fields of the profile) or IP (with the CUSTOM_PARAMETERS field of the profile).

For search/retrieval authentication by IP, a value must be placed in the CUSTOM_PARAMETERS tag from the EBSCOEIT* source profile. Again, this is used to connect to and search a certain database. Using this IP authentication, you may be authenticated to a number of databases.

Example :
AUTH_TYPE=ip;IP_PROFILE=eit;IP_ADDRESS=1.2.3.4
where : IP_ADDRESS=your IP which authenticates to EIT.

Please note that your EBSCO account must be specifically enabled in order to use of EIT. The user/pw used for connecting to http://search.ebscohost.com/ will not work for EIT. Also note that the IP_PROFILE will be either eit or eitws; — this string matches the 3rd section of your 3-part EIT authentication string (ex. s123456.main.eit or s123456.main.eitws). Please check with EBSCO if you are not sure which it is.

2) The aforementioned authentication by IP or user/pw gives us access to the EBSCOEIT API, but it does not provide successful link navigation on the record links obtained.

For this, the PROXY_HOST and PROXY_PORT in the profile must be configured. IMPORTANT: authentication to EBSCO for link navigation is only done by IP.

Example :
proxy.you.com
9797

So, in conclusion, 2 IP authentication settings are needed: one (user/pw or IP) for accessing and searching the EBSCOEIT database desired and the other one (proxy IP) to successfully navigate on the record links obtained.

Questions about your EIT account status should be referred to Gregory Julien [GregoryJulien@ebscohost.com].

Categories: Muse Search, Sources

The instructions and code provided below are based on the following assumptions:

  • MuseProxyFoundationHMAC is the Muse Proxy application configured with HMAC authentication;
  • quiet is the value of the secret;
  • userName and timestamp are the signature parameters;
  • SHA256 is the algorithm;
  • the separator between the signature parameters is . .

Integrate the following code into your ASPX page:

<%@ Page Language="C#" %>
<%@ Import Namespace="System" %>
<%@ Import Namespace="System.Linq" %>
<%@ Import Namespace="System.Security.Cryptography" %>
<%@ Import Namespace="System.Text" %>
<%@ Import Namespace="System.IO" %>
<%@ Import Namespace="System.Collections.Generic" %>

<script runat=server>
public String getHmacURL(String museProxyURL, String applicationID, String generatedHmacDigest, String parametersToSend) {
return museProxyURL + "/" + applicationID + "?sig=" + generatedHmacDigest + parametersToSend;
}
public String getParametersToCreateURL(Dictionary usedParameters){
String parametersToSend = "";
foreach( KeyValuePair param in usedParameters )
{
String key = param.Key;
if (!key.Equals ("userAddress") && !key.Equals ("userAgent") && !key.Equals ("referer")) {
parametersToSend += "&" + key + "=" + param.Value;
}
}
return parametersToSend;
}
public String getValueForGenerateDigest(Dictionary usedParameters, String separator){
String value = "";
int length = usedParameters.Count;
for (int i = 0; i < length; i++) { if (i < length - 1) { value += usedParameters.Values.ElementAt(i) + separator; } else { value += usedParameters.Values.ElementAt(i); } } return value; } public String generateHmacDigest(String algorithm, String secret, String value){ byte[] key = System.Text.ASCIIEncoding.Default.GetBytes(secret); byte[] byteArray = Encoding.ASCII.GetBytes(value); MemoryStream stream = new MemoryStream(byteArray); String digest = null; if (algorithm.Equals ("sha1")) { HMACSHA1 hmacSHA1 = new HMACSHA1 (key); digest = hmacSHA1.ComputeHash (stream).Aggregate ("", (s, e) => s + String.Format ("{0:x2}", e), s => s);
} else if (algorithm.Equals ("md5")) {
HMACMD5 hmacMD5 = new HMACMD5 (key);
digest = hmacMD5.ComputeHash (stream).Aggregate ("", (s, e) => s + String.Format ("{0:x2}", e), s => s);
} else if (algorithm.Equals ("sha256")) {
HMACSHA256 hmacSHA256 = new HMACSHA256 (key);
digest = hmacSHA256.ComputeHash (stream).Aggregate ("", (s, e) => s + String.Format ("{0:x2}", e), s => s);
} else if (algorithm.Equals ("sha384")) {
HMACSHA384 hmacSHA384 = new HMACSHA384 (key);
digest = hmacSHA384.ComputeHash (stream).Aggregate ("", (s, e) => s + String.Format ("{0:x2}", e), s => s);
} else if(algorithm.Equals ("sha512")){
HMACSHA512 hmacSHA512 = new HMACSHA512 (key);
digest = hmacSHA512.ComputeHash (stream).Aggregate ("", (s, e) => s + String.Format ("{0:x2}", e), s => s);
}
return digest;
}
public Dictionary initUsedParameters(){
// timestamp represent the current UNIX timestamp
long ticks = DateTime.UtcNow.Ticks - DateTime.Parse("01/01/1970 00:00:00").Ticks;
ticks /= 10000000; //Convert windows ticks to seconds
String timestamp = ticks.ToString();
// referer is the referer from request
String referer = Request.Url.GetLeftPart(UriPartial.Authority);
// userAddres is IP adress for user from request
String userAddress = GetIP();
// userAgent is userAgent from request header
String userAgent = HttpContext.Current.Request.UserAgent;

Dictionary parameters = new Dictionary();
parameters.Add ("userName", "username");
parameters.Add ("ts", timestamp);
//parameters.Add("referer", referer);
//parameters.Add("userAddress", userAddress);
//parameters.Add("userAgent", userAgent);
return parameters;
}


String algorithm = "sha256";
String secret = "quiet";
String proxyURL = "http://MUSE_PROXY_HOST:PORT";
String applicationID = "MuseProxyFoundationHMAC";
String separator = ".";

public String getDigest(){
String value = getValueForGenerateDigest(initUsedParameters(), separator);
return generateHmacDigest(algorithm,secret,value);
}

public String getURL(){
return getHmacURL(proxyURL, applicationID, getDigest(), getParametersToCreateURL(initUsedParameters()));
}


public static String GetIP()
{
String ip = HttpContext.Current.Request.ServerVariables["HTTP_X_FORWARDED_FOR"];

if (string.IsNullOrEmpty(ip))
{
ip = HttpContext.Current.Request.ServerVariables["REMOTE_ADDR"];
}
return ip;
}



<%=getURL()%>


where replace MUSE_PROXY_HOST:PORT with your actual Muse Proxy host and port.

The aditional file MuseProxyHMAC.cs that needs to be integrated into your project can be dowloaded from here.
The commented lines are for the cases when you want to use in the signature the userAgent/referer/userAddress values.
Note that they must be specified in the Muse Proxy as well (in the $MUSE_HOME\proxy\webcontexts\Applications\MuseProxyFoundationHMAC
\profiles\login\ProxyLoginModuleHMAC.xml file).

Categories: Muse Proxy, Usage

The instructions and code provided below are based on the following assumptions:
– MuseProxyFoundationHMAC is the Muse Proxy application configured with HMAC authentication;
– quiet is the value of the secret;
– userName and timestamp are the signature parameters;
– SHA256 is the algorithm;
– the separator between the signature parameters is . .

Integrate the following code into your Java project:

package com.edulib.muse.proxy.samples;

import java.io.IOException;
import java.io.PrintWriter;
import java.util.LinkedHashMap;

import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

@WebServlet("/index")
public class Index extends HttpServlet {
private static final long serialVersionUID = 1L;

/**
* Default constructor.
*/
public Index() {
// TODO Auto-generated constructor stub
}

/**
* @see HttpServlet#doGet(HttpServletRequest request, HttpServletResponse response)
*/
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
response.setContentType("text/html");

String algorithm = "HmacSHA256";
String secret = "quiet";
String proxyURL = "http://MUSE_PROXY_HOST:PORT";
String applicationID = "MuseProxyFoundationHMAC";
String separator = ".";
// timestamp represent the current UNIX timestamp
String timestamp = "" + System.currentTimeMillis() / 1000L;
// referer is the referer from request
String referer = request.getRequestURL().toString();
// userAddres is IP adress for user from request
String userAddress = request.getRemoteAddr();
// userAgent is userAgent from request header
String userAgent = request.getHeader("user-agent");

LinkedHashMap parameters = new LinkedHashMap();
parameters.put("userName", "test");
parameters.put("ts", timestamp);
parameters.put("referer", referer);
parameters.put("userAddress", userAddress);
parameters.put("userAgent", userAgent);

MuseProxyHMAC museProxyHMAC = new MuseProxyHMAC(proxyURL, applicationID, secret, algorithm, separator, parameters);

PrintWriter out = response.getWriter();
out.print(""<a href=\"" + museProxyHMAC.generatedURL + "\" target=\"_blank\">" + museProxyHMAC.generatedURL + "</a>"");
}
}

where replace MUSE_PROXY_HOST:PORT with your actual Muse Proxy host and port.
The aditional file MuseProxyHMAC.java that needs to be integrated into your Java project can be downloaded from here.

Categories: Muse Proxy, Usage

The overall steps would be:
1) Create the new application as copy of the MuseProxyFoundation template, the ID of the new application to be MuseProxyFoundationHMAC for example.
2) Edit the file

$MUSE_HOME\proxy\webcontexts\Applications\MuseProxyFoundationHMAC\

profiles\AuthenticationGroups.xml

and do the following:
– Locate the

/ICE-CONFIG/AUTHENTICATION_GROUPS/AUTHENTICATION_GROUP/AUTHENTICATIONS

node and remove its content, thus obtaining an empty node:
<AUTHENTICATIONS>
</AUTHENTICATIONS>

– Edit the value of the node

/ICE-CONFIG/AUTHENTICATION_GROUPS/AUTHENTICATION_GROUP/NAME
to be:
HMAC Authentication

– Add the following sequence under the node

/ICE-CONFIG/AUTHENTICATION_GROUPS/AUTHENTICATION_GROUP/AUTHENTICATIONS

<AUTHENTICATION>
<IDENTIFIER>9</IDENTIFIER>
<LEVEL>requisite</LEVEL>
<CLASS>com.edulib.muse.proxy.authentication.modules.ProxyLoginModuleHMAC
</CLASS>
<HANDLER>
<CLASS>com.edulib.muse.proxy.authentication.modules
.ProxyLoginModuleHMACDataHandlerXml</CLASS>
<PARAMETERS>
<CONFIGURATION_FILE>${WEB_CONTEXT_HOME}/profiles/login
/ProxyLoginModuleHMAC.xml</CONFIGURATION_FILE>
</PARAMETERS>
</HANDLER>
</AUTHENTICATION>

(make sure that after pasting the content the XML file is still valid)

3) Refresh the applications properties via the Muse Proxy Administrator Console -> Advanced left menu section -> Operations item -> Refresh Applications button.
Now the HMAC is set with HMAC authentication.

4) Establish and configure the parameters for the HMAC authentication. For this edit the file:
$MUSE_HOME\proxy\webcontexts\Applications\MuseProxyFoundationHMAC
\profiles\login\ProxyLoginModuleHMAC.xml
and make changes according to your requirements. E.g. you may want to change the secret value (default is quiet) and the parameters that you want to hash as part of the signature. By default only the userName (Application ID) and timestamp are used, however you can add the userAgent and/or referer and/or userAddress to be hashed.

We assume for the examples purposes that all defaults remain (e.g. the quiet secret and userName.timestamp as message to sign with HmacSHA1).

Assuming that you want to proxify an URL (ex. http://www.amazon.com/) for the MuseProxyFoundationHMAC Muse Proxy application, the generated HMAC URL will look like:

http://MUSE_PROXY_HOST:PORT/MuseProxyFoundationHMAC?userName=MuseProxyFoundationHMAC
&ts=1469524141&sig=ee5a160dbd37c4867e34e6147a3421d2289bec14
&qurl=http%3A%2F%2Fwww.amazon.com%2F

where MUSE_PROXY_HOST:PORT are the Muse Proxy server details.

Note that by default the validity of this URL is 30 seconds.

For more detailed information on enabling and configuring HMAC authentication refer to the Muse Proxy Advanced Configuration.pdf manual, 6.4.5.8 ProxyLoginModuleHMAC chapter.

5) Create your server side implementation that will generate dynamically the HMAC link(s).

Notes:

1) The generated HMAC URL will work only for 30 seconds (configurable in the value of the TS_EXPIRY field in
$MUSE_HOME\proxy\webcontexts\Applications\MuseProxyFoundationHMAC
\profiles\login\ProxyLoginModuleHMAC.xml)

2) The server generating the HMAC links and the Muse Proxy server must be time synchronized. This is a must, otherwise if the 2 machines are not synchronized with regard to the time, the HMAC links will not work due to the validity value of the signature.
3) If you create proxified links, the destination URL (e.g the value of the qurl parameter) must be URL encoded.

Categories: Muse Proxy, Usage

Load More

Latest

There can be cases when you want that an Application Entry Point takes the end-user directly to the remote source without rewriting. For this a boolean flag is used for redirecting after the last but one request by dropping the proxy prefix and parameters and redirecting to the last URL value (after filling in the existent variables, if any) from the source profile.

The following directive must be added into the source’s XML profile:

true

[for more details check the Muse Proxy Sources Profiling.pdf manual, chapter 6.22 Redirecting to Remote Source]

Categories: General, Muse Proxy

Follow the instructions from bellow for configuring the authentication for the Muse Search Application with Microsoft’s Azure Active Directory, using SAML.
In this scenario, the Muse Search Application is the Service Provider (SP), while Azure AD is the Identity Provider (IDP).

  1. Generate the Service Provider Metadata

    Access the administration end point for SAML at an URL of the form:
    https://your_domain/muse/saml/web/metadata
    where replace your_domain with the actual domain of the Muse installation.
    Use admin as the username and the configured password.

    In the Muse SSO Metadata Administration page click on the Generate new service provider button to access the metadata generation page.
    In the Metadata Generation page, make sure the Signing key and Encryption key values are the proper ones. Fill in an alias value in the Entity alias input. The rest of configurations should be left with the default values. When done click the Generate metadata button.

    In the Metadata Details page follow the steps listed in section “In order to permanently store the metadata follow these instructions:“.
    After the restart of the Muse web service, access again the Muse SSO Metadata Administration page, the metadata details for the newly added entity and click the Download entity metadata button to download it.

  2. Setup a new application in the Azure Portal

    Access the Azure Portal at: https://portal.azure.com/ and navigate to Azure Active Directory -> Enterprise applications from the menu and click New Application. Add the custom application by accessing Create your own application link and add the name of the application (e.g. Muse Search), making sure the option Integrate any other application you don’t find in the gallery (Non-gallery) is selected. Then chose the application created and select Setup single sign-on from the Manage menu, and then click the “SAML” button to access the configuration guide.

    In the new page click the Upload metadata file and select the metadata file which was downloaded at the previous step. All necessary values will be loaded in the Basic SAML Configuration, click the Save button to store them.

    From the Set up Single Sign-On with SAML page, copy the value of App Federation Metadata Url to be used at the next step.

    Make sure that the necessary user groups and users are configured to access the newly created Azure application. See the below instructions for doing this:
    https://learn.microsoft.com/en-us/azure/active-directory/manage-apps/add-application-portal-assign-users

  3. Generate a new IDP

    Access the Muse SSO Metadata Administration pages (as described at #1) and click the Add new identity provider metadata button.
    In the Add New IDP Metadata files page, enter the URL copied from the Azure Portal in the Take metadata from URL: input and click the Test IDP metadata button.

    Next follow the instructions listed in the page to finalize the setup of the new IDP entity.

  4. Wire the Muse Search Application for SAML authentication

    Follow the instruction from below to finalize the setup of SAML authentication with Microsoft Azure AD.

    – Edit the ${MUSE_HOME}/aas/jaas.config file and add at the end of the file the below entry:

    MuseKnowledge.Azure {
    com.edulib.ice.security.authentication.ICELoginModulePropertiesExtractor requisite config="${ICE_HOME}/profiles/ICELoginModulePropertiesExtractor.Azure.xml";
    com.edulib.ice.security.authentication.ICELoginModule requisite ;
    };

    – Download this file and place it in the following location ${USE_HOME}/profiles/ . Edit the downloaded file, locate the following placeholders: PLACE_HERE_THE_IDP_ENTITY_ID, MuseSearchApplicationID and MuseSearchApplicationPassword and replace them with the appropriate values.
    – Test the integration using an URL of the form:
    https://your_domain/muse/servlet/MusePeer/logon/alias/ALIAS?action=logon&userID=MuseKnowledge.Azure&templateFile=passThrough.html&errorTemplate=logon/logon.html&reuseSession=true&idp=IDP_ENTITY_ID
    where replace your_domain, ALIAS and IDP_ENTITY_ID with the appropriate values.

Statistics are offered as a service to Muse Proxy customers, no matter they are locally hosted or on cloud. The statistics are generated from content of the Muse Proxy log files:
– access log (access.log).
– statistics log (MuseProxyStatistics.log).
More details about the Muse Proxy log files in the Muse Proxy Advanced Configuration manual, chapter “2.9 Log Files“.
For the instances hosted by MuseGlobal, there is nothing to be done, we take care of everything.
For the locally hosted instances, the main requirement is to upload from your server on a daily basis the 2 log files – access.log and MuseProxyStatistics.log – corresponding to the previous day, onto our FTP repository – ftp.museglobal.com – from where we will further pick them up for processing into the statistics platform.
The log files upload is carried out by means of scripts driven by the system’s scheduler/cron, depending on the Operating System. We provide such scripts.
The main requirement for the upload scripts is to have a single log file per day and with the date stamp in their filenames, e.g. access-yyyyMMdd.log and MuseProxyStatistics-yyyyMMdd.log. Refer to this FAQ for making the required logging changes.

Download the patch containing the necessary scripts from here. Copy the downloaded archive into ${MUSE_HOME} and extract it. The content is deployed into the following location on disk: ${MUSE_HOME}/proxy/tools.

Windows OS

Make sure that the patch is already deployed.

  • Edit the MuseProxyLogsUpload.xml file and replace the PLACE_HERE_THE_FTP_USERNAME and PLACE_HERE_THE_PASSWORD placeholders with the actual values as provided by the MuseGlobal Support Team.
  • Open a CMD window, change directory to %MUSE_HOME%/proxy/tools/ and run the MuseProxyLogsUploadAnt.bat script. This is for test purposes to make sure the upload script works.
  • Create a scheduler job in Windows to run the %MUSE_HOME%/proxy/tools/MuseProxyLogsUploadAnt.bat script each day after midnight, for example at 1:00 AM. There are many online tutorials for setting up a basic task, like for example here.

Linux OS

Make sure that the patch is already deployed.

  • Edit the MuseProxyLogsUpload.xml file and replace the PLACE_HERE_THE_FTP_USERNAME and PLACE_HERE_THE_PASSWORD placeholders with the actual values as provided by the MuseGlobal Support Team.
  • Open a shell session on the server, change directory to ${MUSE_HOME}/proxy/tools/ and change the permissions of the MuseProxyLogsUploadAnt.sh file to allow execution. Make the same for the ${MUSE_HOME}/proxy/tools/apache-ant-1.10.13/bin/ant file.
  • Edit the MuseProxyLogsUploadAnt.sh and make sure the value for MUSE_HOME points to the correct location on disk of Muse Proxy.
  • Run the MuseProxyLogsUploadAnt.sh script. This is for test purposes to make sure the upload script works.
  • Create a cron job to run the ${MUSE_HOME}/proxy/tools/MuseProxyLogsUploadAnt.sh script each day after midnight, for example at 1:00 AM.
Categories: General, Muse Proxy

The default logger configuration of Muse Proxy is to produce *.log.1, *.log.2*.log.10 files which rotate by size when reaching 10485760 bytes or at midnight, depending on whichever condition is first met:


NOTICE
com.edulib.ice.util.log.ICETextLog
${MUSE_HOME}/proxy/logs/MuseProxy.log
10485760
{0, date,yyyy-MM-dd'T'HH:mm:ss.SSS z} {1}: {4}: {3}
10
24



NOTICE
com.edulib.ice.util.log.ICETextLog
${MUSE_HOME}/proxy/logs/access.log
10485760
%h %A %w %W %u %S %t "%r" "%{Content-Type}o" %s %b "%{User-Agent}i"
10
24



STATISTICS
com.edulib.ice.util.log.ICETextLog
${MUSE_HOME}/proxy/logs/MuseProxyStatistics.log
10485760
{0, date,yyyy-MM-dd'T'HH:mm:ss.SSS z} {3}
10
24



Follow the instructions below to have a single log file with an entire day of logging and with the date stamp in the filename, e.g. MuseProxy-20230719:

  1. Stop the Muse Proxy service
  2. Edit the Muse Proxy main configuration file:
    ${MUSE_HOME}/proxy/MuseProxy.xml
    (make a backup copy first of this file)
    and replace the existing logger sections (see above), with:


    NOTICE
    com.edulib.ice.util.log.ICETextLog
    ${MUSE_HOME}/proxy/logs/MuseProxy.log
    0
    {0, date,yyyy-MM-dd'T'HH:mm:ss.SSS z} {1}: {4}: {3}
    365
    0



    NOTICE
    com.edulib.ice.util.log.ICETextLog
    ${MUSE_HOME}/proxy/logs/access.log
    0
    %h %A %w %W "%u" %S %t "%r" "%{Content-Type}o" %s %b %e "%{User-Agent}i" "%{Referer}i" %D
    90
    0



    STATISTICS
    com.edulib.ice.util.log.ICETextLog
    ${MUSE_HOME}/proxy/logs/MuseProxyStatistics.log
    0
    {0, date,yyyy-MM-dd'T'HH:mm:ss.SSS z} {3}
    365
    0


    Attention must be paid when editing to not break the XML file format.

  3. Start the Muse Proxy service

To make sure Muse Proxy rewrites a website protected by Cloudflare the most reliable way is for the publisher to list the outbound IP address at Cloudflare. This is a must especially if the Cloduflare challenge is set to Interactive Challenge (the captcha-like one).

For publishers not intending to support rewriting proxies, if the publisher site is set up with a non-interactive JS challenge, then the Muse Proxy source profile can be configured using a base64 filter:


Use corresponding URL patterns here


Also some Cloudflare cases may require the following profile settings
true
TLSv1.3
TLSv1.3

Categories: General, Muse Proxy

Load More