Thumbs-up and thumbs-down feedback - IBM watsonx Orchestrate ADK

Enable thumbs-up and thumbs-down feedback in the embedded web chat to gather user opinions on the responses provided by the agent. This feedback mechanism helps improve the quality of interactions by allowing users to express their satisfaction or dissatisfaction with specific messages.

What’s New: Feedback is now enabled by default and automatically stored on watsonx Orchestrate servers.Since embedded chat is not versioned, changes are applied automatically with each release and there is no explicit upgrade or rollout process. As a result, existing implementations may experience breaking changes.To adapt to this update, developers can choose one of the following approaches:

Using wxO storage
Using custom storage

Refer to the sections below for implementation details.

Default behavior

Thumbs-up and thumbs-down feedback is enabled by default in the embedded web chat. Feedback buttons automatically appear on agent responses without requiring any configuration.

Automatic storage on watsonx Orchestrate servers

All feedback is automatically stored on watsonx Orchestrate servers without requiring any special effort. The system handles feedback persistence automatically, including:

Feedback type (thumbs-up or thumbs-down)
Message context and content
Session information and conversation history
Timestamp
User-provided details (categories and comments)

State is maintained: Feedback history and context are preserved in the conversation thread, making it easy to track and analyze user satisfaction over time.

Toggling feedback on or off

While feedback is enabled by default, you can disable it if needed using both pre:receive and pre:threadLoaded event handlers. This ensures feedback is disabled for both new messages and previously loaded messages in the thread:

JavaScript

<script>
    function preThreadLoadedHandler(event, instance) {
        const totalMessages = event.messages.length;

        console.log(`Total messages in thread: ${totalMessages}`);

        // Process only the last message
        const messagesToProcess =
            totalMessages > 0 ? [totalMessages - 1] : [];

        messagesToProcess.forEach((messageIndex) => {
            const message = event.messages[messageIndex];

            if (message && message.content) {
                // Get the last content item in this message
                const lastContentItem =
                    message.content[message.content.length - 1];

                console.log(
                    `Applying feedback to message ${messageIndex}, content item:`,
                    lastContentItem
                );

                if (lastContentItem) {
                    lastContentItem.message_options = {
                        feedback: {
                            is_on: false,
                        },
                    };
                }
            }
        });
    }

    function preReceiveHandler(event) {
        console.log("pre-receive event", event);

        const lastItem =
            event?.message?.content?.[
                event.message.content.length - 1
            ];

        if (lastItem) {
            lastItem.message_options = {
                feedback: {
                    is_on: false,
                },
            };
        }
    }

    function onChatLoad(instance) {
        instance.on("pre:receive", preReceiveHandler);
        instance.on("pre:threadLoaded", preThreadLoadedHandler);
    }

    window.wxOConfiguration = {
        onLoad: onChatLoad,
        // ... other configuration
    };

    setTimeout(function () {
        const script = document.createElement('script');
        script.src = 'https://web-chat.global.assistant.watson.appdomain.cloud/versions/' +
                     (window.wxOConfiguration.clientVersion || 'latest') +
                     '/WatsonAssistantChatEntry.js';
        document.head.appendChild(script);
    }, 0);
</script>

Important: You need both handlers to fully disable feedback:

pre:receive - Disables feedback for new incoming messages
pre:threadLoaded - Disables feedback for previously loaded messages when the thread is restored

Customizing feedback options

You can customize the feedback experience by configuring categories and disclaimers for both positive and negative feedback:

JavaScript

function preReceiveHandler(event) {
    const lastItem = event?.message?.content?.[event.message.content.length - 1];
    if (lastItem) {
        lastItem.message_options = {
            feedback: {
                is_on: true,
                show_positive_details: false,
                show_negative_details: true,
                // Positive feedback categories (only shown if show_positive_details is true)
                positive_options: {
                    categories: ['Helpful', 'Accurate', 'Clear', 'Complete'],
                    disclaimer: "Your feedback helps us improve. Content may be shared publicly.",
                },
                // Negative feedback categories
                negative_options: {
                    categories: ['Inaccurate', 'Incomplete', 'Too long', 'Irrelevant', 'Confusing', 'Other'],
                    disclaimer: "Please provide specific details to help us improve.",
                },
            },
        };
    }
}

function onChatLoad(instance) {
    instance.on('pre:receive', preReceiveHandler);
}

window.wxOConfiguration = {
    onLoad: onChatLoad,
    // ... other configuration
};

State management and context preservation

The feedback system automatically preserves conversation state and context to ensure meaningful feedback analysis. Each feedback submission includes:

Preserved context information

Message ID: Unique identifier for the specific message receiving feedback
Session ID: Conversation session identifier for tracking feedback across interactions
Message content: The actual response text that received feedback
Conversation history: Previous messages in the conversation thread
Timestamp: When the feedback was submitted
User metadata: Any available user information (if configured)

Accessing feedback context

When handling feedback events, you can access the full context:

JavaScript

function feedbackHandler(event) {
    console.log('Feedback received:', {
        feedbackType: event.feedback.type,        // 'positive' or 'negative'
        messageId: event.messageId,               // Unique message identifier
        sessionId: event.sessionId,               // Session identifier
        messageContent: event.message,            // The message that received feedback
        categories: event.feedback.categories,    // Selected categories (if any)
        comment: event.feedback.comment,          // User comment (if provided)
        timestamp: event.timestamp,               // When feedback was submitted
        conversationContext: event.context        // Full conversation context
    });
}

function onChatLoad(instance) {
    instance.on('feedback', feedbackHandler);
}

This preserved state makes it easy to:

Understand the context in which feedback was provided
Correlate feedback with specific agent responses
Track feedback patterns across conversations
Analyze feedback in relation to conversation flow

Using webhook events for custom processing

While feedback is automatically stored on wxO servers, webhook events are still available for additional custom processing. You can subscribe to the feedback event to perform additional actions such as:

Sending notifications to your team
Triggering custom analytics
Integrating with third-party systems
Implementing custom business logic

JavaScript

function feedbackHandler(event) {
    // Feedback is already stored on wxO servers
    
    // Perform additional custom processing
    console.log('Feedback received:', event);
    
    // Example: Send notification to your team
    notifyTeam(event.feedback);
    
    // Example: Track in your analytics system
    trackAnalytics('feedback_submitted', {
        type: event.feedback.type,
        messageId: event.messageId
    });
}

function onChatLoad(instance) {
    instance.on('feedback', feedbackHandler);
}

Routing feedback to custom storage (optional)

While feedback is automatically stored on wxO servers, you can optionally redirect or duplicate feedback to your own custom storage solution for additional processing or integration with existing systems.

Why use custom storage?

Custom storage is useful when you need to:

Integrate feedback with existing analytics platforms
Apply custom data retention policies
Perform real-time feedback analysis
Combine feedback with other application data
Maintain feedback data in your own infrastructure

Implementing custom storage

To route feedback to custom storage, subscribe to the feedback event and send the data to your backend API:

JavaScript

function feedbackHandler(event) {
    // Extract feedback data
    const feedbackData = {
        feedback: {
            type: event.feedback.type,              // 'positive' or 'negative'
            categories: event.feedback.categories,   // Selected categories
            comment: event.feedback.comment          // User comment
        },
        messageId: event.messageId,
        sessionId: event.sessionId,
        message: event.message,
        timestamp: new Date().toISOString(),
        conversationContext: event.context
    };

    // Send to your custom backend
    fetch('https://your-api.example.com/api/feedback', {
        method: 'POST',
        headers: {
            'Content-Type': 'application/json',
            'Authorization': `Bearer ${yourAuthToken}`
        },
        body: JSON.stringify(feedbackData)
    })
    .then(response => {
        if (response.ok) {
            console.log('Feedback stored successfully');
        } else {
            console.error('Failed to store feedback:', response.statusText);
        }
    })
    .catch(error => {
        console.error('Error storing feedback:', error);
    });
}

function onChatLoad(instance) {
    instance.on('feedback', feedbackHandler);
}

window.wxOConfiguration = {
    onLoad: onChatLoad,
    // ... other configuration
};

Custom storage with error handling

For production environments, implement robust error handling and retry logic:

JavaScript

async function feedbackHandler(event) {
    const feedbackData = {
        feedback: event.feedback,
        messageId: event.messageId,
        sessionId: event.sessionId,
        message: event.message,
        timestamp: new Date().toISOString(),
        context: event.context
    };

    try {
        const response = await fetch('https://your-api.example.com/api/feedback', {
            method: 'POST',
            headers: {
                'Content-Type': 'application/json',
                'Authorization': `Bearer ${yourAuthToken}`
            },
            body: JSON.stringify(feedbackData)
        });

        if (!response.ok) {
            throw new Error(`HTTP error! status: ${response.status}`);
        }

        const result = await response.json();
        console.log('Feedback stored:', result);
        
    } catch (error) {
        console.error('Failed to store feedback:', error);
        // Feedback is still stored on wxO servers even if custom storage fails
    }
}

Migration guide for existing customers

If you’re upgrading from a previous version, here’s what has changed:

Previous behavior (before this update)

Feedback was OFF by default in embed mode
Feedback was NOT stored on the backend
State was NOT maintained - no conversation history or context preservation
Custom storage implementation was required to persist feedback

New behavior (current version)

Feedback is ON by default in embed mode (matching platform chat behavior)
Feedback is automatically stored on wxO backend
State is maintained - feedback history and context are preserved in the conversation thread
Webhook events still available for additional custom processing
Custom storage is now optional rather than required

What this means for you

If you had custom storage
If you want to disable feedback
If you had no feedback

Your custom storage will continue to work, but it’s no longer required for basic feedback functionality.Options:

Keep your custom storage - Webhook events still fire, so your existing implementation continues to work
Remove custom storage - Rely on automatic wxO storage and simplify your implementation
Use both - Keep custom storage for additional processing while benefiting from automatic wxO storage

No breaking changes - Your existing code will continue to function as before.

If you prefer to keep feedback disabled (previous default behavior), use both pre:receive and pre:threadLoaded handlers:

JavaScript

<script>
    function preThreadLoadedHandler(event, instance) {
        const totalMessages = event.messages.length;
        const messagesToProcess = totalMessages > 0 ? [totalMessages - 1] : [];

        messagesToProcess.forEach((messageIndex) => {
            const message = event.messages[messageIndex];
            if (message && message.content) {
                const lastContentItem = message.content[message.content.length - 1];
                if (lastContentItem) {
                    lastContentItem.message_options = {
                        feedback: { is_on: false }
                    };
                }
            }
        });
    }

    function preReceiveHandler(event) {
        const lastItem = event?.message?.content?.[event.message.content.length - 1];
        if (lastItem) {
            lastItem.message_options = {
                feedback: { is_on: false }
            };
        }
    }

    function onChatLoad(instance) {
        instance.on("pre:receive", preReceiveHandler);
        instance.on("pre:threadLoaded", preThreadLoadedHandler);
    }

    window.wxOConfiguration = {
        onLoad: onChatLoad,
        // ... other configuration
    };
</script>

Note: Both handlers are required to fully disable feedback for new and previously loaded messages.

Feedback is now enabled automatically. If you don’t want this:

Use the pre:receive handler to disable it (see “If you want to disable feedback” tab)
Or simply leave it enabled - users will see thumbs-up/down buttons on agent responses

No action required if you want to enable feedback.

Important considerations

Embed chat response modifications

Important: Embed chat responses might sometimes be modified at the customer application end before being displayed to users. When implementing custom storage:

The message content stored in feedback may differ from what was originally generated by the agent
Custom formatting, sanitization, or content filtering applied by your application will be reflected in the feedback
Ensure your feedback analysis accounts for these modifications
Consider storing both the original agent response and the modified version for accurate analysis

Data privacy and compliance

When implementing custom storage:

Ensure compliance with data privacy regulations (GDPR, CCPA, etc.)
Implement appropriate data retention policies
Secure feedback data with encryption in transit and at rest
Obtain necessary user consent for feedback collection
Provide users with access to their feedback data if required
Implement data deletion mechanisms as needed

Performance considerations

Implement asynchronous feedback submission to avoid blocking the UI
Use batching for high-volume feedback scenarios
Consider implementing a queue system for reliable delivery
Monitor API response times and implement timeouts
Cache authentication tokens to reduce overhead

Complete implementation example

Here’s a complete example combining all features:

JavaScript

<script>
// Configuration
const FEEDBACK_API_URL = 'https://your-api.example.com/api/feedback';
const AUTH_TOKEN = 'your-auth-token';

// Feedback handler with optional custom storage
async function feedbackHandler(event) {
    console.log('Feedback received:', event);
    
    // Feedback is already stored on wxO servers
    
    // Optional: Send to custom storage for additional processing
    const feedbackData = {
        feedback: {
            type: event.feedback.type,
            categories: event.feedback.categories || [],
            comment: event.feedback.comment || ''
        },
        messageId: event.messageId,
        sessionId: event.sessionId,
        message: event.message,
        timestamp: new Date().toISOString(),
        context: {
            conversationHistory: event.context?.history || [],
            userAgent: navigator.userAgent,
            pageUrl: window.location.href
        }
    };
    
    // Send to custom storage (optional)
    try {
        const response = await fetch(FEEDBACK_API_URL, {
            method: 'POST',
            headers: {
                'Content-Type': 'application/json',
                'Authorization': `Bearer ${AUTH_TOKEN}`
            },
            body: JSON.stringify(feedbackData)
        });
        
        if (response.ok) {
            console.log('Custom storage successful');
        }
    } catch (error) {
        console.error('Custom storage failed (wxO storage still succeeded):', error);
    }
}

// Pre-receive handler to customize feedback options (optional)
function preReceiveHandler(event) {
    const lastItem = event?.message?.content?.[event.message.content.length - 1];
    
    if (lastItem) {
        lastItem.message_options = {
            feedback: {
                is_on: true,  // Feedback enabled by default (can be set to false to disable)
                show_positive_details: false,
                show_negative_details: true,
                positive_options: {
                    categories: ['Helpful', 'Accurate', 'Clear'],
                    disclaimer: "Your feedback helps us improve."
                },
                negative_options: {
                    categories: ['Inaccurate', 'Incomplete', 'Unclear', 'Other'],
                    disclaimer: "Please provide details to help us improve."
                }
            }
        };
    }
}

// Initialize chat
function onChatLoad(instance) {
    instance.on('pre:receive', preReceiveHandler);
    instance.on('feedback', feedbackHandler);
}

window.wxOConfiguration = {
    onLoad: onChatLoad,
    // ... other configuration options
};

// Load the web chat
setTimeout(function () {
    const script = document.createElement('script');
    script.src = 'https://web-chat.global.assistant.watson.appdomain.cloud/versions/' + 
                 (window.wxOConfiguration.clientVersion || 'latest') + 
                 '/WatsonAssistantChatEntry.js';
    document.head.appendChild(script);
}, 0);
</script>

For additional customization options and event handling, see the Events documentation.

​Default behavior

​Automatic storage on watsonx Orchestrate servers

​Toggling feedback on or off

​Customizing feedback options

​State management and context preservation

​Preserved context information

​Accessing feedback context

​Using webhook events for custom processing

​Routing feedback to custom storage (optional)

​Why use custom storage?

​Implementing custom storage

​Custom storage with error handling

​Migration guide for existing customers

​Previous behavior (before this update)

​New behavior (current version)

​What this means for you

​Important considerations

​Embed chat response modifications

​Data privacy and compliance

​Performance considerations

​Complete implementation example

Default behavior

Automatic storage on watsonx Orchestrate servers

Toggling feedback on or off

Customizing feedback options

State management and context preservation

Preserved context information

Accessing feedback context

Using webhook events for custom processing

Routing feedback to custom storage (optional)

Why use custom storage?

Implementing custom storage

Custom storage with error handling

Migration guide for existing customers

Previous behavior (before this update)

New behavior (current version)

What this means for you

Important considerations

Embed chat response modifications

Data privacy and compliance

Performance considerations

Complete implementation example