Database Modeling with Microsoft® Visio for Enterprise Architects (The Morgan Kaufmann Series in Dat

Filtering Data Using Parameters

---

This number denotes the period of time remaining before the failure. In this method, labels are continuous variables. For regression, labeling is done with reference to a failure point. Its calculation is not possible without knowing how long the asset has survived before a failure. So in contrast to binary classification, assets without any failures in the data cannot be used for modeling. This issue is best addressed by another statistical technique called Survival Analysis.

But potential complications may arise when applying this technique to PdM use cases that involve time-varying data with frequent intervals. For more information on Survival Analysis, see this one-pager. In this method, the target variable holds categorical values. Labeling for multi-class classification for failure time prediction.

In this method also, labels are categorical See Figure 6. Labeling for multi-class classification for root cause prediction. The model assigns a failure probability due to each P i as well as the probability of no failure.

These probabilities can be ordered by magnitude to allow prediction of the problems that are most likely to occur in the future. It is just predicting the most likely root cause once the failure has already happened. The Team Data Science Process provides a full coverage of the model train-test-validate cycle. This section discusses aspects unique to PdM.

The goal of cross validation is to define a data set to "test" the model in the training phase. This data set is called the validation set. This technique helps limit problems like overfitting and gives an insight on how the model will generalize to an independent data set. That is, an unknown data set, which could be from a real problem. The training and testing routine for PdM needs to take into account the time varying aspects to better generalize on unseen future data.

Many machine learning algorithms depend on a number of hyperparameters that can change the model performance significantly. The optimal values of these hyperparameters are not computed automatically when training the model. They should be specified by the data scientist. There are several ways of finding good values of hyperparameters. The most common one is k-fold cross-validation that splits the examples randomly into k folds.

For each set of hyperparameters values, run the learning algorithm k times.

iTunes is the world's easiest way to organize and add to your digital media collection.

At each iteration, use the examples in the current fold as a validation set, and the rest of the examples as a training set. Train the algorithm over training examples and compute the performance metrics over validation examples. At the end of this loop, compute the average of k performance metrics. For each set of hyperparameter values, choose the ones that have the best average performance.

The task of choosing hyperparameters is often experimental in nature. In PdM problems, data is recorded as a time series of events that come from several data sources. These records may be ordered according to the time of labeling. Hence, if the dataset is split randomly into training and validation set, some of the training examples may be later in time than some of validation examples. Future performance of hyperparameter values will be estimated based on some data that arrived before model was trained. These estimations might be overly optimistic, especially if the time-series is not stationary and evolves over time.

As a result, the chosen hyperparameter values might be suboptimal. The recommended way is to split the examples into training and validation set in a time-dependent manner, where all validation examples are later in time than all training examples. For each set of hyperparameter values, train the algorithm over the training data set. Choose hyperparameter values that show the best performance.

The final model can be generated by training a learning algorithm over entire training data using the best hyperparameter values. Once a model is built, an estimate of its future performance on new data is required.

A good estimate is the performance metric of hyperparameter values computed over the validation set, or an average performance metric computed from cross-validation. These estimations are often overly optimistic. The business might often have some additional guidelines on how they would like to test the model. The recommended way for PdM is to split the examples into training, validation, and test data sets in a time-dependent manner. All test examples should be later in time than all the training and validation examples.

Azure AI guide for predictive maintenance solutions

After the split, generate the model and measure its performance as described earlier. When time-series are stationary and easy to predict, both random and time-dependent approaches generate similar estimations of future performance. This section describes best practices to implement time-dependent split. A time-dependent two-way split between training and test sets is described below. Assume a stream of timestamped events such as measurements from various sensors. Define features and labels of training and test examples over time frames that contain multiple events.

For example, for binary classification, create features based on past events, and create labels based on future events within "X" units of time in the future see the sections on feature engineering and modeling techniques.

A LOAD DATA Diagnostic Utility

Thus, the labeling time frame of an example comes later than the time frame of its features. For time-dependent split, pick a training cutoff time T c at which to train a model, with hyperparameters tuned using historical data up to T c. To prevent leakage of future labels that are beyond T c into the training data, choose the latest time to label training examples to be X units before T c.

In the example shown in Figure 7, each square represents a record in the data set where features and labels are computed as described above. Time-dependent split for binary classification. The green squares represent records belonging to the time units that can be used for training. Each training example is generated by considering the past three periods for feature generation, and two future periods for labeling before T c. When any part of the two future periods is beyond T c , exclude that example from the training data set because no visibility is assumed beyond T c.

The black squares represent the records of the final labeled data set that should not be used in the training data set, given the above constraint. These records will also not be used in testing data, since they are before T c. In addition, their labeling time frames partially depend on the training time frame, which is not ideal. Training and test data should have separate labeling time frames to prevent label information leakage.

The technique discussed so far allows for overlap between training and testing examples that have timestamps near T c. A solution to achieve greater separation is to exclude examples that are within W time units of T c from the test set. But such an aggressive split depends on ample data availability. Regression models used for predicting RUL are more severely affected by the leakage problem. Using the random split method leads to extreme over-fitting. For regression problems, the split should be such that the records belonging to assets with failures before T c go into the training set.

Records of assets that have failures after the cutoff go into the test set. Another best practice for splitting data for training and testing is to use a split by asset ID. The split should be such that none of the assets used in the training set are used in testing the model performance. Using this approach, a model has a better chance of providing more realistic results with new assets.

In classification problems, if there are more examples of one class than of the others, the data set is said to be imbalanced. Ideally, enough representatives of each class in the training data are preferred to enable differentiation between different classes. The underrepresented class is called a minority class.

Many PdM problems face such imbalanced datasets, where one class is severely underrepresented compared to the other class, or classes. In some situations, the minority class may constitute only 0. Class imbalance is not unique to PdM. Other domains where failures and anomalies are rare occurrences face a similar problem, for examples, fraud detection and network intrusion. These failures make up the minority class examples. With class imbalance in data, performance of most standard learning algorithms is compromised, since they aim to minimize the overall error rate. But the model will mis-classify all positive examples; so even if its accuracy is high, the algorithm is not a useful one.

Consequently, conventional evaluation metrics such as overall accuracy on error rate are insufficient for imbalanced learning. When faced with imbalanced datasets, other metrics are used for model evaluation:. For more information about these metrics, see model evaluation. However, there are some methods that help remedy class imbalance problem. The two major ones are sampling techniques and cost sensitive learning. Imbalanced learning involves the use of sampling methods to modify the training data set to a balanced data set.

Sampling methods are not to be applied to the test set. Although there are several sampling techniques, most straight forward ones are random oversampling and under sampling. Random oversampling involves selecting a random sample from minority class, replicating these examples, and adding them to training data set.

Consequently, the number of examples in minority class is increased, and eventually balance the number of examples of different classes. A drawback of oversampling is that multiple instances of certain examples can cause the classifier to become too specific, leading to over-fitting. The model may show high training accuracy, but its performance on unseen test data may be suboptimal.

Conversely, random under sampling is selecting a random sample from a majority class and removing those examples from training data set. However, removing examples from majority class may cause the classifier to miss important concepts pertaining to the majority class. Hybrid sampling where minority class is over-sampled and majority class is under-sampled at the same time is another viable approach. There are many sophisticated sampling techniques. The technique chosen depends on the data properties and results of iterative experiments by the data scientist.

In PdM, failures that constitute the minority class are of more interest than normal examples. So the focus is mainly on the algorithm's performance on failures. Incorrectly predicting a positive class as a negative class can cost more than vice-versa. This situation is commonly referred as unequal loss or asymmetric cost of mis-classifying elements to different classes. The ideal classifier should deliver high prediction accuracy over the minority class, without compromising on the accuracy for the majority class.

There are multiple ways to achieve this balance. To mitigate the problem of unequal loss, assign a high cost to mis-classification of the minority class, and try to minimize the overall cost. Algorithms like SVMs Support Vector Machines adopt this method inherently, by allowing cost of positive and negative examples to be specified during training. Similarly, boosting methods such as boosted decision trees usually show good performance with imbalanced data. Mis-classification is a significant problem for PdM scenarios where the cost of false alarms to the business is high.

For instance, a decision to ground an aircraft based on an incorrect prediction of engine failure can disrupt schedules and travel plans. Taking a machine offline from an assembly line can lead to loss of revenue. So model evaluation with the right performance metrics against new test data is critical. The benefit the data science exercise is realized only when the trained model is made operational. That is, the model must be deployed into the business systems to make predictions based on new, previously unseen, data. The new data must exactly conform to the model signature of the trained model in two ways:.

The above process is stated in many ways in academic and industry literature. But all the following statements mean the same thing:. As stated earlier, model operationalization for PdM is different from its peers. Scenarios involving anomaly detection and failure detection typically implement online scoring also called real time scoring. Here, the model scores each incoming record, and returns a prediction. For anomaly detection, the prediction is an indication that an anomaly occurred Example: For failure detection, it would be the type or class of failure.

In contrast, PdM involves batch scoring. To conform to the model signature, the features in the new data must be engineered in the same manner as the training data. For the large datasets that is typical for new data, features are aggregated over time windows and scored in batch. Batch scoring is typically done in distributed systems like Spark or Azure Batch. There are a couple of alternatives - both suboptimal:. The final section of this guide provides a list of PdM solution templates, tutorials, and experiments implemented in Azure. These PdM applications can be deployed into an Azure subscription within minutes in some cases.

They can be used as proof-of-concept demos, sandboxes to experiment with alternatives, or accelerators for actual production implementations. These different samples will be rolled into this solution template over time. The Azure AI learning path for predictive maintenance provides the training material for a deeper understanding of the concepts and math behind the algorithms and techniques used in PdM problems. Our new feedback system is built on GitHub Issues.

Read about this change in our blog post. Data Science guide overview and target audience The first half of this guide describes typical business problems, the benefits of implementing PdM to address these problems, and lists some common use cases. Business case for predictive maintenance a business decision maker BDM looking to reduce downtime and operational costs, and improve utilization of equipment Data Science for predictive maintenance a technical decision maker TDM evaluating PdM technologies to understand the unique data processing and AI requirements for predictive maintenance Solution templates for predictive maintenance a software architect or AI Developer looking to quickly stand up a demo or a proof-of-concept Training resources for predictive maintenance any or all of the above, and want to learn the foundational concepts behind the data science, tools, and techniques.

Prerequisite knowledge The BDM content does not expect the reader to have any prior data science knowledge. Business case for predictive maintenance Businesses require critical equipment to be running at peak efficiency and utilization to realize their return on capital investments. By default, most businesses rely on corrective maintenance , where parts are replaced as and when they fail.

Corrective maintenance ensures parts are used completely therefore not wasting component life , but costs the business in downtime, labor, and unscheduled maintenance requirements off hours, or inconvenient locations. At the next level, businesses practice preventive maintenance , where they determine the useful lifespan for a part, and maintain or replace it before a failure. Preventive maintenance avoids unscheduled and catastrophic failures. But the high costs of scheduled downtime, under-utilization of the component before its full lifetime of use, and labor still remain.

The goal of predictive maintenance is to optimize the balance between corrective and preventative maintenance, by enabling just in time replacement of components.

This approach only replaces those components when they are close to a failure. By extending component lifespans compared to preventive maintenance and reducing unscheduled maintenance and labor costs over corrective maintenance , businesses can gain cost savings and competitive advantages. Business problems in PdM Businesses face high operational risk due to unexpected failures and have limited insight into the root cause of problems in complex systems.

Some of the key business questions are: Detect anomalies in equipment or system performance or functionality. Predict whether an asset may fail in the near future. Estimate the remaining useful life of an asset. Identify the main causes of failure of an asset. Identify what maintenance actions need to be done, by when, on an asset.

Typical goal statements from PdM are: Reduce operational risk of mission critical equipment. Increase rate of return on assets by predicting failures before they occur. Control cost of maintenance by enabling just-in-time maintenance operations. Lower customer attrition, improve brand image, and lost sales. Lower inventory costs by reducing inventory levels by predicting the reorder point. Discover patterns connected to various maintenance problems.

Other Books in This Series

Provide KPIs key performance indicators such as health scores for asset conditions. Estimate remaining lifespan of assets. Recommend timely maintenance activities. Enable just in time inventory by estimating order dates for replacement of parts. These goal statements are the starting points for: Qualifying problems for predictive maintenance It is important to emphasize that not all use cases or business problems can be effectively solved by PdM. There are three important qualifying criteria that need to be considered during problem selection: The problem has to be predictive in nature; that is, there should be a target or an outcome to predict.

The problem should also have a clear path of action to prevent failures when they are detected. The problem should have a record of the operational history of the equipment that contains both good and bad outcomes. The set of actions taken to mitigate bad outcomes should also be available as part of these records. Error reports, maintenance logs of performance degradation, repair, and replace logs are also important. In addition, repairs undertaken to improve them, and replacement records are also useful.

The recorded history should be reflected in relevant data that is of sufficient enough quality to support the use case. For more information about data relevance and sufficiency, see Data requirements for predictive maintenance. Finally, the business should have domain experts who have a clear understanding of the problem. They should be aware of the internal processes and practices to be able to help the analyst understand and interpret the data.

They should also be able to make the necessary changes to existing business processes to help collect the right data for the problems, if needed. Failures that cannot be repaired in time may cause flights to be canceled, and disrupt scheduling and operations. PdM solutions can predict the probability of an aircraft being delayed or canceled due to mechanical failures.

Aircraft engine parts failure: Aircraft engine part replacements are among the most common maintenance tasks within the airline industry. Maintenance solutions require careful management of component stock availability, delivery, and planning Being able to gather intelligence on component reliability leads to substantial reduction on investment costs. Finance ATM failure is a common problem within the banking industry.

The problem here is to report the probability that an ATM cash withdrawal transaction gets interrupted due to a paper jam or part failure in the cash dispenser. Based on predictions of transaction failures, ATMs can be serviced proactively to prevent failures from occurring. Rather than allow the machine to fail midway through a transaction, the desirable alternative is to program the machine to deny service based on the prediction.

Energy Wind turbine failures: Wind turbines are the main energy source in environmentally responsible countries, and involve high capital costs. A key component in wind turbines is the generator motor. It is also highly expensive to fix. Failure probabilities will inform technicians to monitor turbines that are likely to fail soon, and schedule time-based maintenance regimes. Predictive models provide insights into different factors that contribute to the failure, which helps technicians better understand the root causes of problems.

Distribution of electricity to homes and businesses requires power lines to be operational at all times to guarantee energy delivery. Circuit breakers help limit or avoid damage to power lines during overloading or adverse weather conditions.

A LOAD DATA Diagnostic Utility | Importing and Exporting Data

The business problem here is to predict circuit breaker failures. PdM solutions help reduce repair costs and increase the lifespan of equipment such as circuit breakers. They help improve the quality of the power network by reducing unexpected failures and service interruptions. Transportation and logistics Elevator door failures: Large elevator companies provide a full stack service for millions of functional elevators around the world. Elevator safety, reliability, and uptime are the main concerns for their customers. These companies track these and various other attributes via sensors, to help them with corrective and preventive maintenance.

In an elevator, the most prominent customer problem is malfunctioning elevator doors. The business problem in this case is to provide a knowledge base predictive application that predicts the potential causes of door failures. Elevators are capital investments for potentially a year lifespan. So each potential sale can be highly competitive; hence expectations for service and support are high.

Predictive maintenance can provide these companies with an advantage over their competitors in their product and service offerings. Wheel failures account for half of all train derailments and cost billions to the global rail industry. Wheel failures also cause rails to deteriorate, sometimes causing the rail to break prematurely.

Rail breaks lead to catastrophic events such as derailments. To avoid such instances, railways monitor the performance of wheels and replace them in a preventive manner. The business problem here is the prediction of wheel failures. To let the user provide more than one value as an answer to a question, enable the Allow Multiple Values check box option. The set of three radio buttons for Discrete Values, Range Values, or Discrete And Range Values tells Crystal whether the user will be choosing or typing a single value which is known as a discrete value or providing a starting range and ending range for data values.

Business case for predictive maintenance

The option to do both provides ultimate flexibility to the user, letting them choose to add a single value at a time as well as ranges of data and to intermix the two ways of choosing values. You can provide a list of values for the user to choose from; this is known as a default value list in Crystal. This lets the user add additional values for the report being run but does not permanently add values to the set of value choices. This gives the person using the report a bit of a head start on choosing valid values.

You can create the list of valid values in three ways:. When you click the Set Default Values button, the dialog shown in Figure 7. The options available on the Set Default Values dialog are dependent on the value type of the parameter. To manually add a valid value, type it into the Select Or Enter Value To Add box and then use the right arrow button to move the value to the Default Values list.

This is useful for adding values like None or Type something here. Setting default values for a parameter. To present a list based on values stored in a database column, use the Browse Table and Browse Field drop-down boxes at the top-left corner of the dialog. These drop-down boxes let you choose the table and field to use to populate the list of values available on the left side of the screen; then you can add valid values to the list on the right by using the arrow buttons between the two lists. It is the list on the right that is seen by the person using the report.

If you already have the values stored externally in a text file, you can use the Import Pick List button to retrieve the values from the file directly into the Default Values list on the right side of the dialog. You can also export the values of a Crystal Default Values list and store it in an external file using the Export Pick List button. The Define Description button can be used to add descriptive text for the user to see that might better explain the value that they are choosing.

The Default Values list is what the user will see when prompted for a parameter value. Crystal will not change or automatically update the list of values even if you chose the option to browse a table and a field. Adding values or deleting values is done manually in the Set Default Values dialog. Sometimes, though, you need to allow direct editing. To help prevent errors in this case, you can use a set of options at the bottom left of the Set Default Values dialog to do the following:. For edit masking, Crystal uses characters to represent valid values in specific positions.

For instance, an edit mask of - - would require that the user type three numbers then a dash then three numbers and a dash and finally four numbers, like this: Edit masking helps prevent bad data by checking the values for validity and commonsense values.

Escape character used to allow A, a, 0, 9, , L,? Data Type Variations The options available on the Set Default Values dialog are dependent on the value type of the parameter. For example, string data allows edit masking while number fields do not. Using a Parameter Field With a parameter field created, you can now put it to work in a report. Since parameter fields appear in the Field Explorer beneath the Parameter Fields category, you can treat the field like any other field, even dragging and dropping it into your report. Parameter fields can be used in any of the following ways:. The Select Expert and Parameters One of the main purposes of using a parameter field in a report is to ask the user a question so that data can be fetched based on the criteria they specify.

To get a parameter field to ask for a value from the person using the report, you need to enter it in the Select Expert. The dialog in Figure 7. Adding a parameter to a report query. When you click the New button, the Choose Field dialog shown in Figure 7. Choosing a comparison field. After choosing a field, you must then choose the comparison operation. Comparison operations like less than , greater than , and equal to appear in the first drop-down box in the Select Expert.

For a complete discussion of the options here, refer back to Table 7. Country and the list of comparison operations available for the data type of the field. Choosing a comparison operator. A second drop-down box will appear after choosing the comparison operator. In this second drop-down box, the parameter field shows up on the top of the list of available values. Parameter field names display with a question mark as the first character and then whatever you named the field.

The list of values in the Select Expert sorts parameter fields to the top since the question mark sorts alphabetically before the letter A. Choosing a comparison value. If you prompt for new values, a dialog similar to that shown in Figure 7. Ask the user a question or two. The Single-Screen Input Approach Crystal uses a single-screen approach to asking for all parameter input. It uses this single screen regardless of how many parameters the report requires. The input screen has three distinct areas, starting at the top:.

• Unsettling Cities: Movement/Settlement (Understanding Cities)
• A Souls Learning
• Conquering Meltdowns - Outsmarting the Stresses in Your Life
• Barrons ESL Guide to American Business English
• Movie Night: Tales of Unknown Collection
• The Retreat of the Elephants: An Environmental History of China
• Teamwork II - Dog Training Manual for People with Disabilities (Service Exercises)