## Step 1: Find Unique Symbols and Probabilities First, get the unique symbols and their probabilities using `unique`, `diff`, and `find` (without any loops): ```matlab % v is the input vector [s, ~, idx] = unique(v); % Unique symbols and their indices M = numel(s); % Number of unique symbols count = diff([; find(diff(sort(idx))); numel(idx)]); % Count occurrences p = count / numel(v); % Probability of each symbol ``` **Explanation:** - `unique(v)` finds the unique symbols and assigns each entry in `v` a label (index). - Sorting `idx` and then using `diff` and `find` allows us to count the number of occurrences of each symbol without loops. - Dividing by the length of `v` gives the empirical probabilities. --- ## Step 2: Compute Entropy Now, calculate the entropy using the probabilities: ```matlab H = -sum(p .* log2(p)); % Entropy formula ``` **Explanation:** - This applies the entropy formula: \( H_1 = -\sum_{k=1}^{M} p_k \log_2 p_k \). - `sum` is allowed (as per instructions). --- ## Final Answer: Complete Function ```matlab function [H, M] = entropy(v) [~, ~, idx] = unique(v); M = max(idx); count = diff([; find(diff(sort(idx))); numel(idx)]); p = count / numel(v); H = -sum(p .* log2(p)); end ``` **This function computes the entropy without using any loops, and uses only `sort`, `diff`, `find`, and `sum` as required.**

# Solution to Problem 6.7 ## Step 1: Find Unique Symbols and Probabilities To calculate the empirical probabilities of each symbol in the input vector \( v \), we utilize the `unique`, `diff`, and `find` functions without any loops. ```matlab % v is the input vector [s, ~, idx] = unique(v); % Get unique symbols and their indices M = numel(s); % Count of unique symbols count = diff([0; find(diff(sort(idx))); numel(idx)]); % Count occurrences p = count / numel(v); % Calculate probability for each unique symbol ``` **Explanation:** - The `unique` function retrieves unique symbols and their corresponding indices. - The `sort` function organizes the indices to facilitate counting with `diff`. - The `diff` and `find` operations determine the count of each symbol, which is then normalized by the total length of vector \( v \) to derive the probabilities. --- ## Step 2: Compute Entropy Next, we compute the entropy \( H \) using the empirical probabilities calculated in Step 1. ```matlab H = -sum(p .* log2(p)); % Entropy calculation ``` **Explanation:** - The entropy formula is applied here: \[ H = -\sum_{k=1}^{M} p_k \log_2 p_k \] - The `sum` function aggregates the contributions of each symbol's probability to the total entropy. --- ## Final Answer: Complete Function for 6.7 ```matlab function [H, M] = entropy(v) [s, ~, idx] = unique(v); % Unique symbols M = numel(s); % Number of unique symbols count = diff([0; find(diff(sort(idx))); numel(idx)]); % Count occurrences p = count / numel(v); % Compute probabilities H = -sum(p .* log2(p)); % Calculate entropy end ``` # Solution to Problem 3.5 The problem states that the first-order entropy \( H \) for a set of \( M \) distinct symbols, where symbol \( k \) is produced with probability \( p_k \), is defined as: \[ H = -\sum_{k=1}^{M} p_k \log_2 p_k \] ## Step 1: Define the Symbols and Probabilities Given \( M \) distinct symbols, we compute their probabilities \( p_k \) empirically from the data source. ## Step 2: Calculate the Entropy Using the formula for entropy, we can directly compute it based on the observed probabilities from our data source. **Entropy Interpretation:** - \( H = 0 \) indicates no uncertainty (only one symbol). - If all symbols have equal probability, \( H = \log_2(M) \). --- ## Final Answer for 3.5 The entropy can be computed with the same function defined in 6.7, replacing \( v \) with the data source that generates the symbols. The function will yield the entropy based on the empirical probabilities derived from the unique symbols.